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The  theme  for  this  year's  conference,  "Readiness  Through  Simulation,"  has 
been  selected  not  only  because  it  supports  the  bicentennial  theme,  but  also 
because  of  the  vital  role  that  training  plays  in  maintaining  the  operational 
readiness  of  our  armed  forces.  This  year’s  theme  also  focuses  attention  not 
only  on  what  research,  development,  and  acquisition  of  training  systems  will 
mean  in  terms  of  maintaining  operational  readiness  for  weapons  systems  cur- 
rently under  development,  but  also  because  it  stresses  the  substantial  role 
that  the  existing  inventory  plays  in  supporting  weapons  systems  which  are 
currently  in  the  field.  It  also  focuses  attention  on  the  need  for  keeping  an 
aging  inventory  current  with  the  operational  systems  which  they  support  and 
ensuring  their  availability  through  an  adequate  support  posture. 

The  development  of  training  systems  to  support  new  weapons  systems  must 
constantly  be  tested  against  effectiveness  criteria  to  ensure  that  training 
needs  are  satisfied  through  most  cost-effective  systems  based  on  viable  alter- 
natives. There  is  a need  for  a strong  research  and  development  program  to 
ensure  that  risk  areas,  both  with  respect  to  technology  and  methodology  of 
training,  are  fully  explored  before  commitments  are  made  for  the  acquisition 
of  training  equipment. 

It  is  the  goal  of  this  conference,  through  the  papers  which  have  been 
selected  for  presentation,  to  encourage  a constructive  dialogue  to  take  place 
so  that  the  readiness  of  our  operational  forces  can  be  maintained  through 
cost-effective  training  systems. 
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INTRODUCTION  TO  THE  CONFERENCE 


G.  V.  Ami co 

Director  of  Engineering 
Naval  Training  Equipment  Center 


The  theme  of  this  year's  conference, 
"Readiness  Through  Simulation,"  is  fitting 
since  it  emphasizes  the  vital  role  that  train- 
ing, and  particularly  training  eguipment, 
plays  in  a peacetime  environment.  This  theme 
not  only  permits  us  to  objectively  assess  ca- 
pabilities and  shortcomings  of  the  inventory 
of  training  eguipment,  but  also  enables  us  to 
project  capabilities  of  future  systems.  While 
previous  conferences  have  placed  emphasis  on 
product  improvement  and  training  effectiveness 
of  future  training  systems,  I plan  to  assess 
the  capabilities  of  training  systems  already 
in  the  inventory  which  are  located  at  formal 
schools  and  fleet  activities.  This  evalua- 
tion of  the  inventory  will  also  focus  atten- 
tion on  those  factors  which  could  improve  the 
training  effectiveness  and  supportabil ity  of 
these  systems. 

First,  I would  like  to  spend  a few  mo- 
ments presenting  information  on  the  capabil- 
ities which  exist  in  the  present  inventory. 
This  training  capability  is  achieved  through 
a Navy/Marine  Corps  inventory  of  $683  million 
of  training  equipment  representing  2,844 
trainers  with  an  acquisition  value  of  over 
$1,000  each.  1 have  translated  the  extensive 
utilization  data  which  are  collected  and  pro- 
cessed on  290  of  the  major  devices  by  the 
Naval  Training  Equipment  Center  for  the  Chief 
of  Naval  Operations  into  another  domain;  name- 
ly, that  of  equivalent  operational  training, 
including  the  magnitude  of  threat  and  number 
of  simulated  weapons  which  are  fired  in  syn- 
thetic training  exercises. 

These  training  systems  were  designed  to 
provide  individual  operator,  team,  and  task 
force  training  on  one,  or  a combination  of 
today's  complex  weapons  systems.  The  train- 
ing tasks  range  from  cognitive  and  psycho- 
motor skills  through  complex  decision-making 
tasks  including  the  collection  of  intelli- 
gence, evaluation  of  data,  and  deployment  of 
platforms  and  weapons  to  cope  with  threats. 

The  training  systems  provide  instruction  in  a 
controlled  environment  where  learning  situa- 
tions can  be  Manually  or  automatically  adapted 
to  the  proficiency  level  of  the  trainees.  The 
instantaneous  disposition  of  forces  through 
initialization  and  reset  features  of  the  de- 
vices permits  economic  repetition  of  difficult 
and  critical  tasks. 

The  290  devices  covered  in  the  utiliza- 
tion reporting  system  provided  245,055 
hours  of  training  in  FY  76  with  an  average 
availability  of  87  percent.  In  some  cases. 


it  has  been  necessary  to  run  the  trainers 
for  two  shifts  to  accommodate  the  student 
load. 

The  aviation  trainer  inventory  consist- 
ed of  125  major  trainers  which  provided 

140.000  hours  of  flight  simulation  for  21 
types  of  aircraft.  Flight  crews  had  to  cope 
with  over  100,000  emergencies.  Thirty-two 
thousand  carrier  landings  were  made.  There 
were  16,000  engagements  against  37,000  air 
targets  and  over  63,000  air-to-air  missiles 
were  fired.  A total  of  46,000  hours  of  air 
ASW  missions  were  flown  against  15,000  sub- 
marine targets  in  which  96,000  sonobuoys, 

2.000  torpedoes,  and  13,000  depth  charges 
were  expended. 

Moving  to  the  surface  ship  trainers, 
the  statistics  are  equally  impressive. 
Eighty-six  surface  ship  trainers  steamed  for 

140.000  hours.  All  of  this  time  was  devoted 
to  tactical  AAW,  ESM,  and  ASW  training  prob- 
lems. These  surface  ships  were  augmented  by 
54  support  ships  and  34  fixed-wing  aircraft 
and  helicopters.  The  crews  were  defending 
against  air  and  submarine  threats.  The 
threats  consisted  of  34  aircraft  and  40  sub- 
marines. In  these  exercises,  45,000  weapons 
of  various  types  were  fired. 

Submarine  trainers  operated  for  21,000 
hours.  During  this  time  there  were  25,000 
diving  and  surfacing  evolutions.  Crews 
corrected  more  than  50,000  malfunctions. 

Attack  submarines  detected,  tracked,  and 
attacked  10,000  target  submarines  and  fired 

20.000  torpedoes  of  various  types. 

There  is  another  segment  of  the  inven- 
tory which  is  equally  important  but  not  as 
glamorous  as  the  complex  trainers.  These 
items  are  the  individual  and  classroom  train- 
ing aids  and  devices  which  are  used  to  sup- 
port individual  self-paced  or  classroom 
instruction. 

Although  the  record  of  training  com- 
pl  ished  with  the  existing  inventory  is  im- 
pressive, there  are  a number  of  shortcomings 
which  must  be  addressed.  These  shortcomings 
reduce  the  overall  effectiveness  of  the  train- 
ing programs  which  are  being  supported.  There 
are  four  major  problem  areas  associated  with 
the  inventory.  One  or  more  of  these  defi- 
ciencies may  exist  on  any  one  system.  The 
four  areas  are;  (1)  systems  no  longer  meet 
current  training  objectives,  (2)  reliability 
and  supportabil ity  of  training  systems. 
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(3)  inventory  obsolescence  and,  (4)  training 
systems  in  which  Operational  System  Equiva- 
lence Ratio  (OSER)  is  less  than  unity. 

I have  introduced  the  term  OSER  to  iden- 
tify the  problem  associated  with  keeping  train- 
ing systems  current  with  their  operational 
counterparts,  both  with  respect  to  hardware 
and  computer  software.  This  area  is  usually 
referred  to  as  the  configuration  management 
problem. 

The  severity  of  the  problems  enumerated 
above  is  influenced  by  the  age  of  the  inven- 
tory and  the  uniqueness  of  its  composition. 

To  indicate  the  severity  problem,  26  percent 
of  the  inventory  is  over  10  years  old  and  57 
percent  is  over  5 years  old.  The  inventory 
that  is  over  5 years  old  represents  analog  and 
discrete  component  digital  systems  technology 
which  is  difficult  to  maintain.  The  inventory 
is  made  up  of  1,126  different  devices  out  of 
a population  of  2,844  which  further  compli- 
cates life-cycle  support. 

The  statistics  on  device  utilization  are 
impressive.  They  point  out  the  success  of  the 
training  device  planning  and  programming  system. 


It  is  this  demonstrated  performance,  coupled 
with  the  future  potential  of  training  systems, 
that  has  been  a major  factor  in  the  rationale 
which  leads  to  the  emphasis  being  placed  on 
training  device  acquisition  programs  today. 

The  value  of  these  systems  cannot  be  properly 
expressed  by  their  acquisition  value.  The 
true  value  can  only  be  measured  by  the  con- 
tribution they  make  to  operational  readiness 
of  our  forces. 

On  behalf  of  the  Naval  Training  Equip- 
ment Center,  1 would  now  like  to  extend  appre- 
ciation for  the  support  and  cooperation  we 
have  received  from  both  Government  agencies 
and  Industry.  Thirty  papers,  covering  a wide 
range  of  subject  matter  in  the  areas  of  re- 
search, development,  design  and  testing  of 
training  systems  will  be  presented  during  the 
next  three  days.  I would  also  like  to  express 
my  sincere  appreciation  for  the  time  and  ef- 
fort that  the  authors  have  contributed  in  pre- 
paring the  papers  which  will  be  presented. 
These  papers  provide  the  basis  for  achieving 
the  conference  goal;  an  effective  exchange  of 
information  and  ideas  between  sponsors,  users, 
operators,  and  producers  of  training  systems 
so  that  operational  readiness  can  be  improved 
through  simulation. 
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The  U.  S.  Merchant  Fleet  Is  an  essential 
element  of  our  commerce  and  defense.  American 
Flag  vessels  operate  under  very  stringent  regu- 
lations governing  vessel  safety  and  the  protection 
of  life  at  sea.  However,  collisions  and  groundings 
Involve  a significant  number  of  vessels  each  year. 
This  constitutes  a problem  which  warrants  an 
organized  effort  toward  solution. 

Research  to  Identify  causes  and  define 
solutions  to  problems  such  as  collisions  and 
groundings  Is  a natural  application  for  simulation. 
The  conditions  of  an  experiment  can  be  controlled, 
and  results  observed  more  accurately  than  If 
actual  vessels  were  used.  Study  variables  can  be 
changed  over  wide  ranges.  New  equipments  and 
procedures  can  be  Investigated  more  easily  and 
((ulckly  In  a simulator  than  In  the  real  world. 
Potential  risk  situations  can  be  Investigated  with- 
out endangering  either  the  "own  ship”  or  other 
traffic.  Also,  the  cost  of  simulation  Is  generally 
much  less  than  the  cost  of  a comparable  study 
using  full-scale  ships  In  an  actual  harbor  or 
seaway. 

With  this  research  in  mind,  the  U.  S. 
Maritime  Administration  in  mid-1971  authorized 
Sperry  Systems  Management,  a division  of 
Sperry  Rand  Corporation,  to  conduct  studies 
toward  the  development  of  a marine  research 
simulator.  These  studies  affirmed  the  concept 
of  simulation  based  research  and  defined  the 
overall  configuration.  Hardware  and  software 
development  was  authorized  in  mid-1973.  The 
resulting  system  was  delivered  by  Sperry  in 
December  1975,  and  put  in  operation  for  the 
National  Maritime  Research  Center  of  the 
Maritime  Administration. 

The  Computer  Aided  Operations  Re- 
search Facility  (CAORF),  is  designed  to  conduct 
navigational  and  ship  handling  experiments  to 
enhance  the  safety,  productivity,  and  competitive- 
ness of  the  American  Merchant  Marine.  Some  of 
the  system  engineering  aspects  of  CAORF  are 
covered  In  this  paper.  System  requirements  are 
considered  first,  followed  by  discussions  of  the 
functions  of  the  major  subsystems  and  the  overall 
configuration,  and  how  each  contributes  to  ful- 
fillment of  the  system  requirements. 

SYSTEM  REQUIREMENTS 

The  system  requirements  for  a research 
simulator  are  somewhat  broader  than  the  require- 
ments for  a simulator  which  is  used  primarily 
for  training.  Much  of  the  investigation  of  the 


causes  of  collisons  and  groundings,  for  example, 
must  focus  on  the  performance  of  the  crew  on  the 
bridge  and  the  man-machine  Interface  at  the 
bridge,  under  hlgh-stress  maritime  situations. 

The  research  simulator  must: 

1)  Simulate  each  of  the  operational  and 
environmental  factors  which 
significantly  affect  the  performance 
of  the  bridge  crew 

2)  Simulate  each  of  these  factors  with 
sufficient  realism  to  produce  valid 
results 

3)  Provide  flexibility  to  permit  rapid 
changes  in  these  factors 

4)  Include  the  capability  for  recording 
and  storing  information  from  each 
experiment  for  study  and  analysis. 

There  are  other  requirements;  however,  these 
are  among  the  most  critical. 

Referring  to  the  first  of  these  require- 
ments, the  important  elements  of  maritime 
operation  which  affect  the  bridge  crew,  and  which 
should  be  simulated  realistically,  include: 

• Instrumentation  and  the  layout  of 
the  bridge 

• Ship's  performance  and  "feel;"  i.e. , 
its  propulsion  and  steering  dynamics, 
and  hydrodynamic  characteristics 

• Configuration  of  the  harbor  or 
seaway  in  which  the  ship  is  operating 

• Navigation  aids,  cultural  features, 
and  important  landmarks  which  are 
visible  from  the  bridge 

• Other  ship  traffic;  types  of  ships, 
courses,  maneuvers 

• Communications  from  the  "own 
ship's"  engine  room,  other  ship 
traffic,  and  shore  facilities 

• Visibility;  fog,  haze,  day,  or  night 

• Unexpected  breakdowns  of  ship 
equipment;  propulsion,  steering, 
and  electric  power. 

• Color  of  objects  in  the  scene. 
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Referring  to  the  second  requirement, 
realism  of  simulation  requires  that  the  bridge 
crew  be  surrounded  by  realistic  equipment,  and 
that  the  sights  and  sounds  of  the  real  world 
situation  be  adequately  reproduced.  For  a re- 
search simulator  which  focuses  on  crew  perfor- 
mance, realism  also  requires  that  the  "mood"  or 
atmosphere  of  the  real  world  situation  be 
established  and  maintained  throughout  the 
simulation.  For  example,  insofar  as  is  feasible, 
the  approaches  to  the  simulation  area  should  have 
a maritime  decor  similar  to  a ship's  passageways. 
Once  the  simulation  is  started,  it  should  not  be 
interrupted  by  external  disturbances  nor  break- 
downs of  simulator  equipment.  Apart  from  the 
impact  of  availability  on  simulator  operating 
economics,  high  availability  is  an  essential 
requirement  for  adequate  realism. 

The  third  requirement  implies  flexibility 
at  several  levels.  During  an  experiment,  for 
example,  the  simulator  should  be  capable  of 
changes  in  weather  and  visibility,  and  in  the 
number  and  types  of  traffic  ships.  It  should  be 
possible  to  make  changes  such  as  bridge 
arrangement,  dynamic  characteristics  of  "own 
ship, " configuration  of  navigation  aids,  or  minor 
cultural  features,  between  parts  of  an  experiment. 
Major  changes,  in  harbor  configuration,  or  the 
experiment  gaming  area,  should  be  possible  in  a 
reasonably  short  time  between  experiments.  Over 
a long  period  of  time,  it  should  be  possible  to  add 
new  equipments  and  capabilities  to  the  simulator 
in  order  to  permit  evaluation  of  proposed  advances 
in  the  state-of-the-art. 

To  meet  the  fourth  requirement,  the 
simulator  system  should  include,  in  addition  to 
conventional  recording  and  printout  or  plotting 
capabilities,  some  means  for  recreating  and 
observing  portions  of  the  experiment  activities 
for  detailed  study  and  analysis. 

The  configuration  and  operation  of  the 
major  CAORF  elements  are  described  next  to  show 
how  CAORF'  meets  the  four  basic  requirements 
above. 

WHEELHOUSE  AND  RRIDGE 

The  bridge  crew  operates  In  a full-scale 
"own  ship's”  wheelhouse,  fully  equipped  with 
contemporary  instrumentation  and  controls.  The 
engine  order  telegraph,  steering  stand,  and 
other  bridge  instruments  ana  controls  are  current 
commercial  models  selected  after  a survey,  to 
identify  those  types  which  are  In  widespread  use 
on  U.  S.  Flag  vessels.  Standard  marine  radar 
units  are  located  on  the  bridge  (see  Figure  1). 
Wheelhouse  construction  is  such  that  all  bridge 
equipment  is  easily  moveable  so  that  the 
arrangement  can  be  changed.  Units  can  be  added 
or  deleted  when  it  is  desired  to  study  the  effect 
of  a particular  arrangement  or  unit  on  the 
performance  of  the  bridge  crew,  or  on  the 
outcome  of  a particular  situation.  Bow  and  stem 
thrusters  and  twin  screw  controls  are  available. 


Figure  1.  Interior  of  wheelhouse  showing 
bridge  equipment 


The  wheelhouse  is  located  at  the  center 
of  a 60-foot  diameter  circular  theater.  On  the 
walls  of  the  theater,  covering  a 240-degree  field 
of  view,  is  the  scene  which  is  observed  through 
the  wheelhouse  windows  at  the  actual  position  and 
heading  of  the  "own  ship."  As  the  "own  ship" 
moves  in  response  to  the  commands  of  the  bridge 
crew,  the  scene  changes  appropriately.  As  many 
as  six  other  moving  traffic  ships  of  various  types 
can  appear  simultaneously  in  the  scene.  Up  to 
40  traffic  ships  can  appear  simultaneously  in  the 
radar  display,  since  the  radar  can  scan  a much 
greater  area  than  can  be  seen  visually.  The 
bridge  is  equipped  with  intercom  and  sound- 
powered  phones  for  simulated  communication  to 
the  engine  room  and  other  parts  of  "own  ship,  " 
a ship's  whistle,  and  radio  equipment  for 
communication  with  other  traffic  ships  and  shore 
installations. 

Figure  2 shows  the  arrangement  of  the 
wheelhouse  and  the  other  major  elements  of  the 
simulator  within  the  CAORF  building.  The 
building  was  designed  and  delivered  as  part  of 
the  CAORF  system.  It  Increases  the  effectiveness 
of  the  system  by  providing  a benign  physical 
environment,  maritime  decor  to  enhance  the  mood, 
and  an  efficient  arrangement  for  operation  and 
maintenance. 

CENTRAL  DATA  PROCESSOR 

A central  computer  simulates  propulsion 
and  steering  dynamics  and  hydrodynamics.  It 
then  solves  the  equations  of  motion  of  "own  ship" 
in  order  to  update  the  "own  ship's"  position  and 
heading.  The  computer  also  updates  the  positions 
and  headings  of  the  traffic  ships  in  the  experiment. 
These  are  under  the  command  of  the  Experiment 
Controller  who  is  located  in  the  Control  Station, 
separate  from  the  bridge  crew.  The  computer 
sends  the  updated  positions  and  headings  of  all  of 
the  ships  plus  information  on  sunlight  and  weather, 
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Figure  2. 


Cutaway  view  showing  arrangement  ol  elements  of  CAORF  simulator  in  its  building. 


thirty  times  per  second,  to  the  appropriate  bridge 
and  control. station  instruments,  three  other 
dedicated  computers  which  generate  the  visual 
scene,  the  radar  displays  on  the  bridge,  and  a 
map-like  situation  display  in  the  Control  Station. 
The  central  computer  also  coordinates  all 
commands  and  data  flow  throughout  the  simulator. 
It  records  data  for  analysis  and,  if  desired, 
playback  of  any  desired  portions  of  the  experiment. 

CONTROL  STATION 

The  experiment  is  directed  from  the 
Control  Station  (see  Figure  3)  where  the 
Experiment  Controller  and  his  assistants  perform 
the  following  functions: 

• Start  and  stop  the  experiment,  change 
operating  modes  (e.  g. , "freeze” 

or  "playback") 

• Monitor  experiment  status  and  issue 
commands  to  the  bridge  crew  as 
appropriate 

• Control  the  insertion,  motion,  and  the 
deletion  of  traffic  ships  in  the  visual 
scene  and  in  the  radar  presentation 

• Perform  the  functions  of  the  engine 
room  of  the  "own  ship,"  providing 
communications,  and  changing 
propulsion  or  steering  upon  request 
from  the  bridge 

• Provide  voice  and  whistle  commu- 
nications as  appropriate  from  other 


traffic  ships,  other  parts  of 
"own  ship,"  and  shore  Installations 

• Control  the  weather,  haze,  fog,  etc. 

• Insert  simulated  failures  of 

"own  ship' 8"  propulsion,  steering, 
or  electric  power. 

The  Control  Station  is  equipped  with  an 
equivalent  set  of  bridge  instruments  and  controls. 
Instead  of  a duplicate  radar  presentation,  however, 
there  is  a situation  display  of  equivalent  size  which 
presents  a continually  updated  map  of  the  exper- 
iment gaming  are?.  The  map  shows  shorelines, 
channels  and  buoys,  and  all  traffic  ships  partic- 
ipating in  the  experiment,  with  symbolic 
indications  of  types,  and  vectors  indicating 
heading  and  speed.  The  Experiment  Controller 
has  a choice  of  several  scales  and  modes  of 
presentation. 


Figure  3.  Control  Station 
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The  Control  Station  also  Includes 
appropriate  "own  ship's"  Intercom  and  sound- 
powered  phones,  and  simulated  radio  channels  for 
communications  from  other  traffic  ships  and  the 
shore.  There  is  also  an  interactive  CRT  terminal 
and  keyboard  for  communicating  with  the  central 
computer  and  other  controls  as  necessary  for 
starting  and  terminating  the  experiment,  changing 
operating  modes,  controlling  visibility,  inserting 
simulated  equipment  failures,  and  performing 
other  functions. 

SIGNAL  PROCESSING  PHILOSOPHY 

Digital  signal  transmission  and  process- 
ing is  used  throughout  the  simulator  in  order  to 
meet  the  basic  system  requirements  more 
efficiently.  For  example,  the  signals  to  and  from 
the  bridge  instruments  and  controls  are  converted 
to  digital  form  adjacent  to  the  bridge.  All  signals 
and  commands,  except  voice  communications,  are 
routed  through  the  central  computer.  In  addition 
to  simulating  such  functions  as  ship  responses, 
the  computer  is  able  to  coordinate  and  synchronize 
related  events,  thus  giving  a smooth  and  realistic 
overall  simulation.  Changes  in  many  types  of 
parameters,  such  as  changes  in  ship's  character- 
istics, insertion  and  deletion  of  traffic  ships,  and 
changes  in  types  of  ships,  are  facilitated  by  the 
use  of  digital  data.  This  computer-centered 
configuration  also  facilitates  the  storage  of 
information  from  the  experiment  for  analysis. 

As  many  as  several  hundred  quantities  can  be 
recorded,  if  desired;  the  rate  of  recording  can  be 
selected  by  the  Experiment  Controller  and  varied 
during  an  experiment.  There  is  also  a playback 
mode  where  the  experiment  can  be  frozen  at  any 
time,  and  any  previous  portion  can  be  repeated 
for  instance,  all  simulated  motion  of  "own  ship" 
and  traffic  ships,  visual  and  instrument  displays, 
and  changes  in  propulsion  and  steering  commands 
are  repeated  in  real-time  for  observation  and 
analysis.  Playback  can  be  exercised  during  an 
experiment  or  at  a later  date.  Figure  4 is  a CAORF 
system  functional  flow  diagram. 

In  an  extension  of  this  system  philosophy, 
computer  generated  imagery  (CGI)  is  used  to 
create  the  three  visual  displays;  the  maritime 
scene  in  the  theater  viewed  through  the  wheelhousc 
windows,  the  radar  displays  on  the  bridge,  and  the 
situation  display  in  the  control  station.  Each  of 
these  displays  is  generated  by  a separate, 
dedicated  computer  which  is  under  the  control  of 
the  central  data  processor.  The  use  of  separate 
data  bases  for  the  three  displays  was  found  to  be 
more  efficient.  The  visual  scene  is  the  most 
extensive  of  the  three;  its  operation  will  be 
discussed  next. 

IMAGE  GENERATOR  AND  DISPI.AY 

The  data  base  for  the  visual  scene  is 
stored  in  the  dedicated  computer.  When  the 
position  and  heading  of  the  "own  ship"  is  re- 
ceived from  the  central  data  processor,  the 


dedicated  computer  selects  those  elements  of  the 
data  base  which  would  be  seen  from  the  bridge  of 
"own  ship"  at  that  position  and  heading.  The 
selected  scene  elements  are  processed  to  show  the 
proper  range  and  aspect,  and  to  eliminate  elements 
hidden  by  closer  elements.  The  scene  is  then 
converted  to  TV  raster  format  for  transmission  to 
the  display  projectors.  These  operate  on  the 
Fidophor  principle,  similar  to  those  used  for 
closed-circuit  TV  presentations  in  large  auditoriums. 
Figure  5 shows  a typical  computer  generated  visual 
scene  in  CAORF. 

The  use  of  CGI  greatly  enhances  CAORF' s 
capability  to  simulate  and  change  several  fact  ors 
which  have  an  important  affect  on  crew  performance. 
It  greatly  facilitates  the  inclusion  of  moving  traffic 
ships  In  the  scene.  It  permits  total  unrestricted 
movement  and  perspective  change  of  the  scene  in 
response  to  motions  of  the  simulated  "own  ships.  " 
Also,  the  eye  position  may  be  arbitrarily  located 
in  both  height,  fore,  and  aft  directions.  Thus, 
large  or  small  "own  ships,  " with  bridges  mounted 
either  forward  or  aft,  may  be  simulated.  The  bow 
of  "own  ship"  can  appear  on  the  screen  if 
appropriate;  its  appearance  is  that  of  the  selected 
type  of  "own  ship."  To  date,  two  oil  tankers  and 
one  LNG  tanker  have  been  encoded  for  display. 

Others  can  be  readily  added  as  the  need  occurs. 

The  ambient  illumination  in  the  scene  is 
continuously  variable  from  day  to  night.  In  day- 
time scenes,  the  sun's  position  is  suggested  by 
highlights  and  shading  on  buildings,  terrain,  and 
other  picture  elements.  In  twilight  and  night  scenes, 
the  lights  located  on  ships  and  on  shore  are 
controllable  by  groups.  Each  light  is  represented 
in  its  correct  color,  location,  flashing  pattern, 
and  visibility  range.  Lights  having  a directional 
characteristic  or  color  changes  are  so  represented. 

The  area  used,  in  the  visual  scene,  for 
a single  continuous  exercise  is  normally  50  by 
100  miles  (the  radar  gaming  area  is  appropriately 
larger  to  allow  for  the  greater  range  of  the  radar 
on  a ship  which  is  located  near  the  edge  of  the 
visual  gaming  area).  If  desired,  the  experiment 
may  shift  after  a short  interruption  to  a continuous 
gaming  area  of  equal  size.  The  gaming  area 
represented  in  the  present  data  base  covers  the 
approaches  to  New  York  through  the  narrows  and 
through  the  Kill  Van  Kull  to  Port  Newark.  When 
more  areas  are  encoded,  exchanging  one  magnetic 
disk  in  the  image  generator  computer  will  change 
the  visual  scene  to  represent  a totally  different 
area. 

Within  the  limits  of  computer  capacity, 
the  scene  encoded  into  the  data  base  can  show 
bridges,  piers,  buoys,  buildings,  terrain  features, 
cranes,  and  other  fixed  objects  of  Importance  to 
mariners  navigating  in  the  harbor.  Nautical  charts, 
topographic  maps,  photographs,  and  other  sources 
are  used  to  assure  the  accuracy  of  presentation  as 
well  as  the  priority  of  feature  selection  in  terms 
of  importance  to  the  mariner.  Provision  is  made 
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Figure  5.  Computer  Generated  Maritime  Scene 
viewed  through  wheelhouse  windows 


to  add  or  remove  aids  to  navigation,  or  make  other 
changes  as  warranted  by  Notices  to  Mariners. 

One  of  the  most  dramatic  features  of  the 
visual  system  is  the  presentation  of  fog.  The  image 
generator  computer  "knows”  the  distance  from  the 
viewer  to  each  object  in  the  scene.  In  response  to 
a control  setting,  the  scene  is  modified  by  contrast 
reduction  with  range  to  give  a realistic  impression 
of  fog.  Provision  has  also  been  made  to  show  fog 
banks  in  a realistic  manner  so  that  lights  or 
objects  are  partially  or  totally  obscured  as 
appropriate.  The  fog  bank  may  move  or  ships  may 
move  through  it. 

CONCLUSION 

This  paper  has  covered  some  of  the  sys- 
tem aspects  of  the  CAORF  ship’s  bridge  simulator. 


CAORF  is  designed  for  research  into  shiphan- 
dling problems,  particularly  the  causes  of 
collisions  and  groundings,  and  the  performance 
of  the  bridge  crew  in  stressful  situations.  These 
research  problems  are  a fruitful  application 
which  can  realize  several  of  the  unique  advan- 
tages of  simulation  over  investigations  using  real 
world  equipment.  The  research  simulator  has 
the  capability  to  simulate  critical  operational 
factors,  which  affect  the  outcome  of  the  experi- 
ments, with  sufficient  realism  to  obtain  valid 
crew  performance,  with  the  facility  to  vary  each 
of  the  factors.  CAORF  consists  of  a full-scale 
ship's  bridge,  fully  equipped  with  contemporary 
hardware,  with  digital  handling  of  essentially  all 
information  and  commands,  and  computer  gener- 
ated imagery  to  create  maritime  scenes.  The 
scenes  include  sufficient  numbers  of  moving 
traffic  ships,  weather  effects,  and  harbour 
waterway  configuration  to  simulate  realistic 
problem  situations.  The  cost  of  simulating  ma- 
rine operational  problems  in  this  way  is  consid- 
erably less  than  it  could  be  in  the  actual  vessel. 
Further,  the  simulation  can  include  complex  and 
hazardous  conditions  which  are  out  of  the  ques- 
tion for  operating  vessels.  The  experiments  can 
be  conducted  with  the  advantages  of  laboratory 
tests;  properly  skilled  technicians  recording 
results  on  scenarios  undisturbed  by  unplanned 
intrusions,  and  undeflected  by  the  crisis  which 
may  result  from  such  intrusions.  Simulation 
can  provide  experiments  for  entry  into  channels 
not  yet  dredged,  and  docks  not  yet  built  for 
ships  not  yet  designed.  This  can  yield  data  to 
improve  maritime  systems  and  operations. 
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GENERATION  OF  AUDIO  SIGNALS  FOR  SONAR 
SIMULATION/STIMULATION  WITH  DIGITAL  TECHNIQUES 

MORRIS  H.  STEPHENSON  and  FORREST  D.  SUCHEY 
Honeywell,  Marine  Systems  Division 


ABSTRACT 

Sonar  signals  are  conventionally  divided  into 
two  categories:  (1)  shaped,  broadband  components, 
such  as  ambient,  ship  flow  and  propeller  cavitation 
noises,  and  (2)  narrowband  components,  including 
machinery  noise,  echoes,  and  reverberations. 

This  paper  will  explain  how  Honeywell  uses  digital 
data  processing  to  generate  these  signals  for  use 
in  sonar  trainers. 

BROADBAND  SPECTRUM  SIGNAL  GENERATION 

For  the  purjxjse  of  this  discussion , the 
broadband  spectrum  addressed  is  the  bandwidth  of 
10  Hz  to  20  kHz,  a spectrum  of  interest  in  gener- 
ating audio  signals  used  in  sonar  simulation  and 
stimulation.  The  ocean  spectrum  is  known  to  have 
certain  predominant  shaping  in  this  bandwidth  due 
to  wind,  shipping,  and  ocean  lift. 

The  results  of  recent  ambient-noise  investi- 
gations have  been  published  . The  Knudson 
curves  show  the  ocean  spectrum  shaping  over  the 
frequency  spectrum  ranging  from  1 Hz  to  10^  Hz. 
This  spectrum  may  be  viewed  as  a shaped,  broad- 
band spectrum.  This  section  discusses  an  ap- 
proach used  in  digital  signal  processing  to  synthe- 
size this  spectrum.  The  shaped,  broadband  spec- 
trum can  be  subdivided  into  smaller  bands , from 
which  filter  networks  can  be  designed,  and  the 
overall  filter  outputs  summed  to  create  the  shaped 
broadband  spectrum. 

One  type  of  filter  that  can  be  used  to  develop 
the  broadband  shaping  required  is  a n-pole/m-zero 
recursive  digital  filter  of  the  general  expression: 


The  Read-Only  Memories  provide  a table 
look-up  for  the  filter  coefficients,  Figure  1. 

These  coefficients  also  could  be  a Random  Access 
Memory  (RAM),  which  could  be  computer-loaded 
for  added  filter  flexibility.  The  filter  input  sam- 
ples, Figure  2,  are  then  multiplied  by  the  neces- 
sary coefficient  using  a parallel  digital  multiply 
algorithm  utilizing  MSI  digital  multipliers.  These 
partial  products  are  then  summed  over  the  number 
of  poles  and  zeros  in  the  digital  parallel  adder  to 
meet  the  requirements  of  equation  1.  The  output 
of  the  filter  is  then  sent  to  a gain  adjustment  net- 
work, which  also  can  be  a digital  multiplier. 

The  same  constraints  in  implementing  low- 
pass,  band-pass,  and  high-pass  digital  filters 
also  apply  in  this  form  of  spectrum  shaping,  i.e.  , 
the  sampling  frequency  chosen  must  be  sufficiently 
high  to  prevent  unwanted  aliasing  and  foldover. 

The  coefficient  range  must  be  sufficient  to 
allow  proper  pole  and  zero  placement.  The  num- 
ber of  bits  carried  through  the  filter  must  be  suf- 
ficient to  achieve  the  dynamic  range  and  resolu- 
tion required  by  the  desired  spectrum  shape. 

Once  the  desired  spectrum  shape  has  been 
determined,  a model  can  be  formed  from  which  a 
computer  simulation  can  be  utilized  to  determine 
the  coefficient  sizes.  The  attractiveness  of  this 
type  of  digital  filter  implementation  is  the  flexibi- 
lity afforded  in  the  selection  of  the  number  of  poles 
and  zeros  required  for  a particular  filter  network. 
The  same  digital  multipliers  and  adders  also  can 
be  used  for  all  the  filters.  This  type  of  imple- 
mentation is  a sampled  filter  timesharing  the 
multipliers  and  adders. 
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The  recursive  filter  is  considered  to  be  the 
(1)  most  cost-effective  approach  that  will  achieve  the 

desired  spectrum  shape. 


where:  Y 

out 


One  filter  output 


NARROWBAND  SPECTRUM  SIGNAL  GENERATION 


a.  = Zero  coefficients 

Xj  = Filter  inputs 

b.  Pole  coefficients 

y . = Delayed  outputs 

The  first  stage  in  developing  this  broadband 
spectrum  is  to  develop  a flat  response  over  the 
bandwidth  of  interest.  This  is  achieved  by  using  a 
digital  pseudo- random  number  generator  of  suffi- 
cient length.  The  pseudo-random  generator  must 
then  be  sampled  at  a frequency  sufficiently  high  to 
obtain  the  desired  bandwidth,  and  also  eliminate 
the  unwanted  aliasing  foldover  due  to  the  Nyquist 
constraint. 


The  basic  requirements  for  narrowband 
signals  used  to  simulate  or  stimulate  sonar  equip- 
ment is  to  generate  lines  with  the  following  con- 
trolled characteristics: 


1.  Frequency 

2.  Frequency  slide 

3.  Amplitude 

4.  Time  on/off 

5.  Line  spread 

6.  Delta  phase. 
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Figure  1.  Broadband  Digital  Audio  Model 


As  may  be  seen  from  Figure  3,  the  approach 
illustrated  in  this  paper  Is  to  use  digital  techniques 
to  generate  and  control  these  signals.  First,  a 
sine  wave  is  stored  in  a Read  Only  Memory  (ROM); 
second,  a step-by-step  approximation  of  a sine 
wave  is  generated  by  extracting  sequential  values 
from  the  ROM.  Third,  the  resulting  sine  wave  is 
attenuated  to  the  desired  value  with  a digital  mul- 
tiplier and  summed  with  other  sine  wave  points  to 
provide  a composite  waveform  comprising  several 
sine  waves  of  varying  frequency  and  amplitude. 

There  are  two  basic  methods  for  generating 
sine  waves  with  varying  frequencies;  each  involves 
stepping  through  a ROM  table.  The  first  is  to  step 
through  a table  with  a variable  clock  rate.  The 
second  uses  a fixed  clock  rate  and  step  with  dif- 
ferent step  sizes.  We  have  chosen  to  use  the  lat- 
ter. The  fixed  clock  method  provides  the  benefi- 
cial side  effect  of  more  steps  per  cycle  at  lower 
frequencies.  This  will  be  explained  later. 

To  keep  high  frequency  resolution,  in  addi- 
tion to  the  basic  count  or  step  size,  fractional 
parts  causing  a non-integer  count,  ax-e  carried. 
This  results  in  counts  running,  for  example,  7,7, 
8,  7,  7,  8.  Using  this  technique  required  the 
analysis  of  two  questions: 

1.  What  is  the  harmonic  distortion  caused 
by  carrying  non-integer  counts? 

2.  What  is  the  harmonic  content  of  the 
wave,  based  on  the  fact  that  the  sine  wave  is  being 
approximated  by  discrete  steps  ? 

Analysis  of  both  problems  discloses  that 
(1)  the  harmonic  distortion  due  to  the  fact  that  we 
count  7,  7,8,7,  7,  8 is  44  dB  down;  this  will  not 
create  a problem;  (2)  the  harmonic  content  re- 
sulting from  discrete  point  samples,  approximating 
a sine  wave,  requires  a more  complex  explanation. 
A computer  simulation  shows  that  the  first  set  of 
bad  harmonics  are  n-1  am;  n+1 , with  n being  the 
number  of  samples  per  cycle.  This  means  that  the 
sample  rate  must  be  selected  to  ensure  that  the  n-1 
and  n*l  harmonics  arc  outside  the  band  of  interest. 
Solution  of  this  problem  is  aided  by  the  fact  that, 
as  frequency  decreases  the  number  of  steps  per 
sample  increases.  Consequently,  this  yields  a 
bad  harmonic  content  at  a constant  frequency. 

As  an  example,  we  are  currently  generating  lines 
up  to  5 kHz  with  the  lowest  sample  rate  being  32 
kHz.  The  frequency  of  interest,  in  this  particular 
case  (5  kHz),  results  in  the  harmonic  distortions 
showing  up  outside  of  the  band  of  interest. 

Another  important  effect  to  be  simulated  is  a 
frequency  slide.  For  this  simulation,  the  count  or 
frequency  of  the  generated  wave  is  updated  once 
each  second  by  computer  control.  If  It  becomes 
desirable  to  change  frequency  during  this  one- 
second  period,  such  as  a doppler  rate  change  for  a 
target,  or  a frequency  modulated  pulse  transmis- 
sion, an  additional  control  is  provided;  this  tech- 
nique is  called  delta  count.  With  delta  count,  a 


number  is  added  to  the  count  number  every  itera- 
tion for  selection  of  a point  in  a sine  wave.  In 
this  manner  the  frequency  can  slide  up  or  down. 

This  control  also  is  updated  once  each  second. 

After  generation,  the  sine  wave  is  fed  to  a 
digital  multiplier,  Figure  3,  with  a control  word 
representing  amplitude,  which  is  updated  once 
each  second  from  the  computer.  It  is  then  multi- 
plied (i.  e.  , attenuated)  in  order  to  form  the  final 
desired  amplitude  and  frequency  shaped  sine  wave. 

These  are  the  basic  controls  by  which  nar- 
rowband generation  is  accomplished.  There  are 
other  controls,  however,  that  are  necessary  as  well 
as  some  other  . vents  that  take  place  (e.g.  , an 
active  echo)  that  require  on  times  of  less  than  one 
second.  For  these  reasons,  the  computer  outputs 
another  control  word  that,  in  essence,  will  be 
counted  down  so  that  once  a signal  is  turned  on  it 
can  be  turned  off  in  increments  of  one  millisecond. 

Another  important  factor  of  narrowband  sim- 
ulation is  that  the  waves  should  not  be  pure  sine 
waves  but  should  have  some  spreading  and  fading. 
This  effect  is  simulated  by  amplitude  modulation 
of  the  wave  with  random  numbers  that  have  a 
Rayleigh  distribution. 

This,  in  turn,  will  cause  both  spreading  and 
fading.  By  controlling  the  rate  of  the  random 
number  selection,  the  amount  of  spread  and  fade 
may  be  bordered. 

The  techniques  discussed  above  have  been 
utilized  to  generate  the  narrowband  noise.  Ob- 
viously some  of  the  effects  have  impact  on  the 
system  design.  For  an  example,  as  the  sine 
waves  must  be  any  frequency,  ranging  from  a very 
low  frequency  to  higher  frequencies,  there  can  be 
no  further  filtering  of  the  output  of  the  narrowband 
audio  generator.  The  only  filter  allowed  will  be  a 
final  system  filter  limiting  the  bandwidth  to  that 
of  interest  to  the  sonar  being  simulated.  This,  in 
turn,  sets  the  number  of  points  that  must  be  used 
per  cycle  to  ensure  that  the  harmonic  content  in 
the  band  of  interest  is  low  by  placing  the  higher 
amplitude  harmonics  outside  of  the  band  of  inter- 
est. 

Another  consideration  is  the  resolution 
required  on  the  lines.  Consider,  for  example, 
that  lines,  controllable  in  0. 1 Hz,  are  desired; 
this  dictates  the  number  of  fractional  bits  that 
must  be  carried  with  the  count  word.  A potential 
problem  existed  previously,  because  a non-integer 
number  of  samples  per  cycle  is  used,  with  the 
possible  net  effect  of  creating  subharmonics.  If 
this  were  so,  then  the  effect  would  show  up  on 
spectrum  analyzers  and  cause  operator  distraction. 
A computer  analysis,  however,  definitely  estab- 
lishes that  there  is  no  subharmonic  generation. 

System  considerations  must  take  into  ac- 
count the  required  step  size  to  move  the  undesirea 
harmonics  out  of  the  range  of  interest,  thus  setting 
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the  number  of  samples  per  second  required. 

The  controllable  amplitude  range  and  the  de- 
sired amplitude  steps  will.  In  turn,  establish 
the  number  of  bits  required  to  obtain  range. 

One  final  consideration,  transcending  that 
of  the  simple  line  generator,  Is  the  fact  that  in 
the  undersea  environment  a distinct  possibility 
exists  that  the  same  line  from  a target  will 
arrive  at  the  own  ship  point  over  two  vertical 
paths.  These  will  then  sum  together  to  pro- 
duce interference  effects.  These  effects  are 
simulated  by  generating  a second  set  of  lines, 
at  the  same  frequency  but  differing  In  ampli- 
tude, to  represent  the  second  path  received  by 
the  sonar;  the  second  path  Is  then  phase-adjusted 
relative  to  the  first  In  accordance  with  the  math 
model  that  generates  path  length  or  time  of 
travel  on  the  path. 
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A SYSTEM  ORIENTED  BENCHMARK  FOR  TRAINING  SIMULATORS 


P.  S.  BABEL  and  DR.  M.  L.  B1RNS 
Aeronautical  Systems  Division 
Wright-Patterson  Air  Force  Base 


1.  INTRODUCTION 

An  important  task  in  the  development  of 
training  simulators  is  determining  the  com- 
puting system  which  is  adequate  for  the 
computational  task  required.  The  definition 
of  the  simulator  computational  system  as  well 
as  ensuring  its  adequacy  has  been  rendered 
mo>-e  difficult  by  the  increasing  complexity 
of  available  computer  systems,  the  ever 
changing  computer  systems  market  and  the 
growing  sophistication  of  the  training  simu- 
lator computation  requirement. 

Several  selection  tools  have  been  pre- 
viously available,  but  are  generally  CPU 
rather  than  system  oriented.  Since  these 
computer  selection  tools  do  not  address 
simulation  processing  parameters,  they  are 
of  limited  utility  in  selecting  computer  sys- 
tems for  this  application.  Hence,  the 
Air  Force  is  striving  to  develop  a computa- 
tional system  selection  technique  which  is 
based  on  training  simulator  requirements 
and  is  flexible  enough  to  be  used  in  the 
various  types  of  computational  systems 
presently  found  in  training  simulators.  The 
tool  required  is  a system  benchmark  which 
will  measure  the  total  capabilityof  the 
computational  systems. 

2.  BACKGROUND 

The  digital  computation  systems  origi- 
nally provided  for  training  simulators  were 
single  CPU  computer  systems.  These  com- 
puters were  programmed  in  assembly  language 
and  required  relatively  slow  responses.  By 
relatively  slow,  we  refer  to  a comparison 
with  the  modern  training  simulator  and  the 
necessity  to  drive  an  advanced  visual  system. 

The  design  for  these  computational 
systems  was  initially  based  on  deriving, 
from  the  functional  ^quirements  of  the 
systeih,  the  high-level  flow  chart  for  the 
program  and  estimating  the  numbers  and  types 
of  instructions  required.  Calculation  of  a 
data  base  was  then  added  to  determine  the 
memory  requirements  of  the  system,  the 
speed  being  defined  and  verified  by  the 
estimated  cal culati onal  requirements. 

As  design  became  somewhat  more  advanced, 
some  scientific  mixes  of  instructions  were 
defined  for  the  typical  calculational  pro- 
gram, such  as  the  classical  gibson  mix,  and 
the  adequacy  of  the  computational  power  was 
determined  by  measuring  the  capability  of  a 


CPU  to  handle  the  assumedly  representative 
instructions  repertoire. 

Historically,  hardware  has  been  evalu- 
ated. Williams1,  as  well  as  Stimler  and 
BrunsS  gives  examples  of  techniques  for 
calculating  computer  performance  and  selec- 
ting an  appropriate  machine. 

Lucas^  lists  and  describes  the  various 
techniques  used  for  performance  evaluation: 

Timi  ngs 

Mixes 

Kernel  s 

Models 

Benchmarks 

Synthetic  Programs 

Simulation 

He  concludes  that  simulation  is  the 
optimum  technique,  but  recognizes  disadvan- 
tages, incl  uding  cost.  Boyse  and  Warn,*4  as 
well  as  many  others,  have  described  computer 
modeling  for  performance  prediction.  They 
make  the  coirment,  which  may  be  self-evident, 
but  is  important,  "Modeling  can  be  useful, 
but  any  model  can  easily  produce  invalid 
results  if  its  input  data  or  assumptions  are 
incorrect. " 

Progress  was  made  as  more  experience  was 
gained  by  being  able  to  use  analogy  from  one 
system  to  another.  Based  on  previous 
experience,  it  was  possible  to  get  a gross 
estimate  of  the  numbers  of  instructions  and 
data  required  for  various  simulator  functions 
by  using  approximations  from  one  simulator 
system  to  the  next.  Similarly  somie  bench- 
marks were  derived  based  on  the  actual  soft- 
ware of  previous  simulators.  The  benchmarks 
presently  available  give  some  indication  of 
comparative  machine  power,  but  they  are 
necessarily  limited  in  the  amount  of  func- 
tions they  can  replicate  and  are  in  general 
limited  to  single  CPU  investigations. 

With  the  passage  of  time,  the  aircraft 
for  which  the  Air  Force  must  train  its 
pilots  have  become  increasingly  complex  and 
the  training  simulator  has  followed  them  in 
sophistication.  This  increased  sophistica- 
tion is  hrought  about  not  only  by  the 
increased  complexity  of  the  aircraft,  e g., 
multiple  on-board  computers,  but  has  also 
been  brought  about  by  the  desire  to  provide 
more  sophisticated  training  capability. 
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The  expanded  use  of  software  to  achieve 
these  levels  of  simulation  together  with  a 
significant  increase  in  computer  performance 
capabilities  has  facilitated  use  of  FORTRAN 
and  commercial  software.  Use  of  these 
software  tools  enhances  software  support- 
ability.  Simultaneously,  the  increased 
complexity  of  the  simulation  job  has 
brought  about  the  need  for  multiple  CPU 
computational  systems  which  utilize  shared 
memory,  inter-computer  channels  and  sophisti- 
cated input/output  devices  for  their  record- 
ing as  well  as  occasional  overlays. 

Computer  systems  design  and  verification, 
once  performed  by  manual  computations  aug- 
mented by  heavy  doses  of  intuition,  can  no 
longer  be  handled  satisfactorily  without  an 
automated  tool  which  provides  the  ability  to 
judge  system  capability  objectively.  Note, 
we  talk  about  systems  capability,  not  com- 
puter or  CPU  capability,  since  much  more 
than  the  capability  of  the  CPU  is  involved. 
The  computational  system  must  be  judged  on 
its  practical  capability,  throughput  capa- 
bility, configurational  capability,  as  well 
as  the  important  software  tools  such  as  the 
operating  system,  compiler  and  utilities 
which  form  an  important  part  of  the  total 
computational  systems. 

Lucas  emphasizes  the  importance  of  the 
programming  system  stating,  "Eecause  the 
programming  system  is  an  integral  part  of 
modem  computers,  the  evaluation  process 
must  now  consider  software  as  well  as  hard- 
ware in  assessing  performance." 

Thus,  it  is  no  longer  sufficient  to 
attempt  to  estimate  the  capacity  of  a data 
processing  system  in  terms  of  a number  of 
average  instructions  or  operations.  It  has 
become  essential  to  have  a tool  which 
measures  all  the  critical  system  parameters 
and  which  is  flexible  enough  to  put  appropri- 
ate emphasis  on  those  which  are  most  impor- 
tant for  a specific  training  simulator.  This 
leads  to  the  idea  of  a system  oriented 
training  simulator  benchmark. 

So  called  "natural"  benchmarks  also 
pose  some  problems.  Keller  and  Denham^ 
comment  on  the  difficulty  of  running  a 
benchmark  on  all  systems,  exactly  as  con- 
figured in  proposals.  This,  however,  is 
required  if  valid  results  are  to  be  obtained. 

An  implication  of  the  "system  benchmark" 
concept  is  provided  in  Joslin,t>who  also 
defines  an  Application  Benchmark  (Problem) 
to  be  "a  routine  to  be  run  on  several  dif- 
ferent computer  configurations  to  obtain 
comparative  throughput  performance  figures 
regarding  the  abilities  of  the  various  con- 
figurations to  handle  the  specific  applica- 
tion." This  implies  that  the  specific 


routine  to  be  executed  is  an  integral  part  of 
the  benchmark  problem. 

As  we  are  interested  in  allowing  varying 
approaches  to  a specific  functional  require- 
ment, we  feel  that  it  is  more  appropriate  to 
start  from  the  functional  definition  of  the 
set  of  tasks  to  be  performed.  The  specific 
processing  design  is  assumed  to  be  dependent 
upon  the  computational  system  configuration 
to  be  used. 

The  advantages  of  using  totally  synthetic 
programs  rather  than  natural  programs  are 
discussed  in  Oliver,  et  al.?  Primary  advan- 
tages tare  portability  and  limitation  of 
processing  time.  A disadvantage  is  the  time 
and  cost  required  to  prepare  programs. 

We  intend  to  circumvent  these  difficul- 
ties by  providing  synthetic,  but  representa- 
tive, operational  designs,  implemented  pri- 
marily by  consistent  blocks  of  FORTRAN  code. 
The  constraint  of  FORTRAN  is  imposed  because 
of  the  de  facto  standardization  of  FORTRAN  as 
a training  simulator  language  as  previously 
discussed  by  Babel. 8 

3.  MEASUREMENT 

The  system  benchmark  must  measure  both 
system  parameters  and  machine  parameters  and 
must  weiqht  these  parameters  such  that  the 
results  indicate  the  capability  of  the  system 
to  do  the  entire  job.  System  parameters  are 
those  system  features  which  impact  the  entire 
computational  system,  not  specifically  the 
computational  capability  of  CPU.  The  system 
parameters  include; 

Operating  System  Capability 
Operating  System  Efficiency 
Fortran  Efficiency  (compiled  object 
code) 

I/O  Capabi  li  ty 
Multiprocessing  Capability 
Systems  Extendabi  li  ty 

Naturally,  the  most  important  part  of  the 
benchmark  is  to  measure  the  actual  computa- 
tional power  of  the  system.  This  must  be 
measured  across  more  than  one  CPU  if  a multi- 
CPU  system  is  envisioned.  For  example,  a 
distributed  CPU  system  may  indeed  have  more 
computational  power  than  that  provided  by  a 
single,  larger  CPU.  However,  the  power  of 
the  distributed  system  does  not  increase 
linearly  and  the  benchmark  must  measure  not 
only  the  power  of  the  single  CPU,  but  the 
extent  to  which  processing  power  is  iicreased 
by  additional  CPUs. 

Thus,  the  benchmark  must  examine  such 
traditional  parameters  as  handling  of  specif- 
ic data  types,  arithmetic  capabilities 
(particularly  floating  point),  general 
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functional  capabilities  and  logical  func- 
ional  capabilities.  It  must  also  be  capable 
of  examining  such  nonclassical  capabilities 
as  the  interference  between  CPUs  operating 
on  a shared  memory,  the  possibility  for 
distributing  computational  power,  the  effect 
of  inter-computer  communication,  both  on 
computational  power  and  I/O  capacity. 

Finally,  the  benchmark  must  be  able  to  assess 
the  capability  of  the  system  to  perform  all 
these  functions  in  the  FORTRAN  environment, 
with  respect  to  the  effect  on  both  process- 
ing time,  memory  requirements,  and  code- 
ability  with  proposed  extensions. 

Since  many  of  the  parameters  which  are 
being  measured  are  intimate  features  of  the 
specific  systems  being  investigated,  it  may 
not  be  possible  to  code  a particular  bench- 
mark program  and  execute  it  on  all  machines. 
FORTRAN  extensions  for  different  systems  are 
different,  operating  systems  have  different 
requirements,  and  machi ne-machi ne  comnunica- 
tion  requirements  are  different. 

It  is  proposed,  therefore,  to  develop 
certain  blocks  of  code  from  representative 
simulator  systems.  These  blocks  of  code  will 
be  FORTRAN  coded  and  will  represent  typical 
simulation  tasks.  In  addition,  the  functions 
to  be  performed  by  the  system  will  be  speci- 
fied by  means  of  high-level  logic  descrip- 
tions. 

An  example  of  the  type  of  functional 
benchmark  design  under  consideration  is 
shown  in  block  form  in  figure  1.  Program  A 
operates  on  input/\  to  produce  dataA;  program 
B operates  on  inputs  to  produce  datag.  Pro- 
gram C operates  on  dataA  and  datag  to  produce 
output^;  programg  operates  on  datag  to  pro- 
duce outputg.  As  a further  constraint,  the 
larger  of  output  C or  D will  be  used  for  the 
next  computation. 

Depending  on  the  time  frame  in  which 
output^  and  outputg  are  required,  as  well 
as  the  available  computational  speed,  the 
sequence  shown  in  figure  2 might  be  suffi- 
cient. This,  obviously,  is  a single  CPU 
solution.  However,  in  a compressed  time 
frame,  one  might  execute  programs  A and  C in 
one  machine,  while  executing  programs  B and 
D in  another.  This  would  yield  an  executing 
configuration  very  similar  to  figure  1 
itself. 

A good  benchmark  should  be  able  to 
determine  whether  either  proposed  configura- 
tion can  handle  the  assumed  processing  load 
in  the  context  of  operational  programs' 
FORTRAN  coding,  commercial  system  software 
and  a superimposed  environment  (interrupts, 
I/O,  etc.).  Therefore,  it  is  important  to 
avoid  specifying  a configurational  solution 
in  the  specification  or  the  problem. 


In  order  to  avoid  specifying  the  sequen- 
tial or  parallel  performance  of  various 
functions,  standard  flow  charts,  with  their 
inherently  sequential  flavor,  will  not  be 
used  to  define  benchmark  requirements.  The 
type  of  documentation  under  consideration 
would  be  similar  to  directed  flowgraphs. 

This  will  allow  the  potential  parallelism  of 
the  problem  to  be  presented. 

A directed  flowgraph  for  the  process 
previously  described  is  presented  in  figure 
3.  The  functional  parallelism  is  evident 
and  the  association  with  a single  machine/ 
multiple  machine  implementation  has  been 
removed,  thus  divorcing  the  functional 
requirement  from  the  implementation  design. 

A brief  definition  of  the  symbols  used 
in  this  description  follows.  The  description 
is  taken  from  Rubey,9  which  provides  more 
detailed  information. 

A directed  flowgraph  represents  a com- 
puting machine  in  which  the  operations  are 
represented  by  the  nodes  of  the  graph,  and 
the  transmission  and  storage  of  data  and 
control  information  is  represented  by  con- 
necting links.  Control  links  are  represented 
by  lines  with  open  arrow  heads;  data  links 
have  solid  arrow  heads. 

Several  classes  of  nodes  are  used,  two 
of  which  are  shown  in  figure  4.  Operator 
nodes  represent  a function  of  one  or  more 
inputs.  Data  operators  have  output  connect- 
ors, all  of  which  are  data.  For  example, 
task  nodes  perform  some  operation  on  the 
data  conveyed  by  the  input  data  links  and 
make  the  result  available  on  output  data 
links.  Task  nodes  may  also  have  input  con- 
trol links  which  affect  the  execution  of  the 
node,  but  do  not  affect  the  value  produced 
if  the  execution  proceeds. 

A specific  operator  node  which  is  used 
in  the  example  of  figure  3 is  the  identity 
node,  labeled  "I."  This  is  a data  node 
which  transmits  its  input  data  value  to  all 
data  output  links  with  no  change.  Its 
utility  comes  from  its  use  with  control 
li  nks. 

The  selector  node  is  the  means  by  which 
alternate  sequences  of  operations  may  be 
executed,  based  on  the  value  or  status  of 
some  data.  Selector  nodes  have  at  least  one 
data  input  link,  and  always  have  two  control 
output  links.  The  selector  has  associated 
with  it  a predicate  B,  which  is  applied  to 
the  data  on  the  data  input  links.  If  the 
resulting  value  is  TRUE,  then  the  "4"  con- 
trol output  is  ENABLED  and  the  control 
output  is  DISABLED.  If  the  result  is  FALSE, 
then  the  converse  link  status  results. 
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Figure  2.  Sequential  Data  Flow 
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Figure  1.  Required  Data  Flow 


Considering  figure  3,  we  see  that  the 
functions  delineated  by  the  left  and  right 
hand  processing  chains  now  appear  independent 
from  a processing  point  of  view.  They  can 
be  implemented  in  whatever  fashion  is  most 
appropriate  for  the  particular  proposed  con- 
figuration. The  code  used  for  the  actual 
operations  A,  B,  C,  and  D can  be  prepared  in 
FORTRAN  and  compiled  and  executed  regardless 
of  the  configuration. 

4.  SUftlARY 

Because  of  the  complex  requirements  of 
training  simulators,  as  well  as  the  range  of 
techniques  by  which  these  requirements  can 
be  met,  a semi -automated  tool  is  required  to 
aid  in  system  selection.  This  tool  must 
evaluate  total  computational  system  capa- 
bility; software  as  well  as  hardware. 

This  requirement  is  being  met  by  the 
design  of  a simulation  system  benchmark 
based  on  natural  code  blocks  configured  into 
a set  of  synthetic,  functional  problems.  The 
expression  of  the  problem  will  be  structured 
so  as  to  avoid  any  implication  regarding  the 
implementation  of  the  solution. 
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INCORPORATION  OF  DIGITAL  AVIONICS  SYSTEMS 
IN  THE  B-l  TRAINING  SIMULATOR 

0.  R.  MOYEN-VAN  SLIMMING 
Aeronautical  Systems  Division 


INTRODUCTION 

The  B-l,  which  is  intended  to  replace 
the  B-52  in  the  Strategic  Air  Command  (SAC) 
inventory,  has  several  simulator  requirements, 
ranging  from  study  carrels  to  full  mission 
simulators.  The  aircraft  has  a sophisticated 
avionics  package,  similar  to  that  in  the 
FB-111.  It  controls  the  navigation,  weapon 
delivery  and  defensive  functions  for  the  air- 
craft. It  utilizes  a number  of  analog  and 
digital  computing  devices  whose  functions 
have  an  impact  on  the  simulator.  A detailed 
analysis  of  the  avionics  system  was  accom- 
plished in  order  to  identify  the  problems 
that  may  be  encountered  in  fulfilling  the 
avionics  function  requirements  for  the  crew 
training  simulator  and  indicate  which 
approach(es)  to  implementing  the  avionics 
functions  would  be  most  cost  effective  over 
the  life  cycle  of  the  B-l  simulator.  There 
are  several  simulator  requirements  which  have 
no  equivalents  in  the  aircraft.  One  such 
requirement  is  the  capability  to  freeze  the 
mission,  usually  with  the  intent  to  reset  the 
simulated  aircraft  to  another  position,  or  to 
instruct  the  crew  concerning  a portion  of  the 
mission  without  their  having  to  concentrate 
on  aircraft  status.  Another  such  require- 
ment is  parameter  freeze,  i.e.,  the  ability  to 
freeze  certain  parameters  such  as  fuel  status, 
pitch,  altitude,  etc.  This  allows  the  in- 
structor to  make  a task  easier  by  freezing 
one  or  more  parameters  so  that  the  crew  can 
concentrate  on  other  tasks.  These  require- 
ments represent  a definite  challenge  to  the 
implementation  of  avionics  functions  iri  the 
simulator. 

AIRCRAFT  OVERVIEW 

The  B-l  is  a long  range,  supersonic, 
swing-wing  bomber.  Its  mission  includes 
tanker  rendezvous,  low  level  penetration  at 
high  subsonic  speeds  and  the  weapon  delivery 
of  high  energy  conventional  stores,  nuclear 
gravity  stores,  and  SRAMs.  The  B-l  has  a 
crew  of  four:  pilot,  copilot.  Offensive 

Systems  Operator  (0S0) , and  Defensive  Systems 
Operator  (DSO).  The  0S0  and  DSO  are 
physically  located  separately  from  the  pilot/ 
copilot  in  an  aft  station.  There  are 
presently  provisions  for  two  instructors, 
one  in  the  cockpit  and  one  in  the  aft 
station. 


Pilot/Copilot  Station 

The  most  striking  departure  from 
previous  aircraft  instrumentations  is  the 
Vertical  Situation  Display  (VSD),  essentially 
an  electronic  Attitude  Direction  Indicator 
(ADI).  It  utilizes  a computer  driven  CRT, 
which  can  also  be  used  to  display  Forward 
Looking  Infrared  (FLIR)  information.  In 
production  models  this  CRT  may  also  be  used 
to  provide  offensive  station  data,  such  as 
weapon  status,  to  the  pilot.  Although  the 
traditional  dial-type  instruments  are  still 
present  (particularly  in  the  backup  instru- 
ments) much  of  the  pilot/copilot  information 
is  provided  on  vertical  tape  type  instruments 
and  some  in  digital  format  (such  as  airspeed, 
mach  number,  and  altitude). 

The  B-l  has  a Terrain  Following  Radar 
(TFR)  which  has  a display  between  the  pilot 
and  copilot.  This  system  has  cwo  analog 
computers  which  are  dual  redundant  and  can 
operate  in  a manual  or  automatic  mode. 

In  the  first  three  prototype  aircraft, 
the  escape  system  consists  of  a crew  module; 
however,  in  the  production  models  this  will  be 
replaced  by  individual  ejection  seats.  This 
change  has  necessitated  some  redesign  of  the 
instrument  panels,  particularly  in  the  aft 
station.  This  may  delay  development  of  the 
aft  station  in  the  simulator. 

The  B-l  is  also  equipped  with  an 
automatic  flight  control  system  (AFCS)  which 
provides  a variety  of  pitch  and  roll, 
automatic  guidance  and  automatic  throttle 
control.  This  system  uses  two  analog 
computers  in  dual  redundant  mode. 

Aft  Station 

While  Rockwell  International  has  the 
airframe  responsibility,  Boeing  (Seattle)  and 
AIL  share  responsibility  for  the  aft  station, 
with  Boeing  being  the  overall  avionics 
system  integration  contractor.  The  Offensive 
Station  utilizes  a Forward  Looking  Radar 
(FLR)  for  navigation  and  weapon  delivery  and 
a Forward  Looking  Infrared  System  (FLIR)  for 
use  in  a nuclear  environment.  The  FLIR 
presentation  appears  on  a multi-function 
display  (MFD)  which  is  physically  located 
above  the  FLR  display.  This  display  can  also 
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be  used  as  an  alphanumeric  display  (AND)  in 
conjunction  with  the  other  three  CRTs.  In 
the  normal  mode  twin  ANDs  are  used  to  display 
navigation/weapon  delivery  information,  while 
another  AND  is  used  in  conjunction  with  the 
integrated  keyboard  ( I KB ) to  select  different 
options  using  the  Offensive  Flight  Software 
(OFS)  logic  trees.  In  case  of  failures  of  a 
CRT,  any  of  the  other  three  can  be  used  to 
display  the  required  information.  Navigation 
and  Stores  Management  System  (SMS)  data  are 
displayed  on  the  NAV  and  SMS  panels  respect- 
ively. The  0S0  can  also  steer  the  FLIR 
sensor. 

The  Data  Entry  Unit  (DEU)  is  a cartridge 
tape  device  by  which  the  0S0  can  load 
the  avionics  computers  with  their  computer 
programs  and  mission  data.  He  has  the 
capability  to  erase  classified  data  and  to 
shutdown  the  offensive  computer  complex.  The 
DSO  has  the  same  capability  over  the 
defensive  computers.  Both  operators  have 
access  to  the  Central  Integrated  Test  System 
(CITS)  Control  and  Display  (CCD)  panel,  and 
a few  flight  instruments. 

The  DSO  has  an  I KB  and  CRT  which  are 
identical  to  the  ones  in  the  offensive 
station,  except  that  navigation/weapon 
delivery  functions  are  automatically  locked 
out  (likewise  the  0S0  cannot  enter  any  DSO 
functions  from  his  IKB).  The  DSO  has  two 
electronic  display  units  (EDU)  which  can  be 
used  to  display  the  threat  environment  in 
panoramic  or  situational  format.  The  infor- 
mation displayed  on  the  EDUs  is  entirely 
computer  generated.  Aural  warning  is 
provided  for  immediate  threats  such  as  a 
detected  missile  launch.  Additionally,  the 
DSO  has  the  option  to  request  reconstituted 
audio  for  any  detected  emitter. 

The  defensive  system  is  basically  an 
automatic  system,  but  the  DSO  has  the  option 
to  override  the  system  and  change  jammer 
power,  threat  priorities,  etc.  In  the 
production  aircraft,  the  DSO  will  also  have 
control  over  expendable  ECM  (flares  and 
chaff)  and  possibly  a tail  warning  system. 

ON-BOARD  COMPUTER  SYSTEMS 
SIMULATION  ALTERNATIVES 

The  presence  of  digital  processors  in 
the  aircraft  presents  a challenge  in  the 
development  and  support  of  training 
simulators,  particularly  if  the  processors 
are  software  programmable.  The  question  of 
providing  the  best  way  of  simulating  systems 
which  utilize  such  processors  is  a vital  one, 
since  it  has  a great  impact  on  development 
risk,  performance,  facility  for  change,  and 
on  Initial  and  life  cycle  costs  of  the 
simulator.  It  is  usually  not  advantageous  to 
use  aircraft  equipment  in  the  simulator 
because  of  the  high  cost  of  such  equipment 


and  because  the  simulator  operates  in  a 
simulated  instead  of  a "real-wofld" 
environment.  It  is  neither  necessary  nor 
cost-effective,  for  example,  to  jse  an  air- 
craft operable  UHF  transceiver,  since  any 
ground  communication  is  usually  handled  by 
the  instructor;  hence,  an  inexpensive  intercom 
set  is  sufficient.  Likewise,  it  probably  is 
impractical  to  use  the  nuclear-hardened 
devices  used  in  the  aircraft,  since  the 
simulator  will  not  encounter  an  environment 
which  requires  such  protection. 

With  programmable  digital  devices  an 
additional  consideration  must  be  made,  i.e., 
what  kind  of  a change  activity  is  expected 
over  the  life  cycle  of  the  aircraft  (and 
consequently  of  the  simulator)?  Devices  in 
which  the  program  is  expected  to  change  fre- 
auently  are  usually  core  memory  driven,  while 
devices  in  which  the  software  change  activity 
is  low  frequently  use  Programmable  (lead  Only 
Memories  (PROMs),  which  are  more  rugged  than 
core  memory  but  are  also  more  expensive. 

This  limits  their  use  in  the  simulator. 

There  are  several  alternative  to  simu- 
lating programmable  digital  devices.  The 
following  paragraphs  will  cover  the  approaches 
considered  as  candidates  in  a training 
simulator  application. 

Aircraft  Hardware 

In  this  approach  the  actual  On-Board 
Computers  (OBC)  are  used  in  the  simulator 
with  the  aircraft  Operational  Flight  Program 
(OFP).  These  OBCs  may  be  softer  (non- 
hardened)  versions  of  the  aircraft  computer 
using  non-mil  sDec  parts,  but  still  being 
functionally  identical  to  the  aircraft 
computer.  The  intent  of  this  approach  is  to 
obtain  a hands-off  update  capability  for  the 
simulator.  The  OFP  support  facility  for  the 
aircraft  would  send  a tape,  which  is  identical 
to  the  aircraft  tape,  to  the  simulator  support 
team  whenever  the  OFP  is  modified. 
Theoretically  then,  all  that  would  be 
necessary  to  update  the  simulator  would  be  to 
read  the  new  tape  into  the  OBC. 

Advantages  of  Aircraft  Hardware:  Rapid 

OFP  update  for  all  software-only  changes 
can  be  achieved,  as  well  as  one-to-one 
correspondence  functionally  with  the  OBC, 
thereby  maximizing  realism  of  signal 
stimuli  and  response  to  inputs.  Also,  the 
aircraft  OFP  is  exercised  much  more  in  the 
simulator  than  in  the  actual  aircraft,  thus 
giving  the  possibility  of  finding  bugs  in  the 
OFP  on  the  ground  which  may  not  be  found  in 
the  aircraft.  This,  however,  does  not 
necessarily  improve  the  training  environment. 

Disadvantages  of  Aircraft  Hardware: 

High  recurring  hardware  costs  are  incurred 
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and  hardware  life-cycle  costs  may  be  high 
also,  since  any  aircraft  computer  change  will 
have  to  be  accomplished  in  the  simulator. 

The  interface  between  the  aircraft  computer 
and  its  environment  in  the  simulator  must  be 
designed  one-to-one,  i.e.,  signals  must  be 
represented  exactly  (timing,  amplitudes,  etc.) 
as  in  the  aircraft.  This  interface  is  complex 
and  expensive.  Simulator  peculiar  functions, 
such  as  freeze,  may  be  difficult  to  implement, 
since  the  simulator  must  use  the  unmodified 
OFP.  Particularly  , parameter  freeze  is  diffi- 
cult to  implement.  One  suggested  method  is 
to  include  simulator  peculiar  functions  in  the 
aircraft  OFP.  This  would  eliminate  one  of  the 
advantages  of  simulation,  i.e.,  the  flexibility 
to  implement  any  training  requirements 
independent  of  the  aircraft  design  implemen- 
tation. Additionally,  any  future  trainer 
peculiar  requirements  may  require  further  OFP 
modifications.  This  would  increase  the  size 
of  the  aircraft  OFP  without  attendant  benefits 
to  the  flying  mission  and  may  preclude  mission 
essential  changes.  Another  problem  would 
arise  if  a new  OFP  change  comes  in  and  it 
does  not  play  in  the  simulator.  The  fault 
may  be  in  the  OFP  itself,  in  the  simulator, 
or  in  the  interface,  and  may  be  difficult  to 
resolve  since  the  OFP  support  team  is  not 
knowledgeable  in  the  detailed  design  of  the 
simulator,  and  the  simulator  support  team  is 
not  necessarily  knowledgeable  in  the  details 
of  OFP  development.  Close  coordination  of 
these  two  software  support  teams  would  lessen 
the  problem  but  would  not  eliminate  it 
entirely. 

Hardware  Emulator 

The  emulator  approach  requires  a micro- 
programmable  general  purpose  (GP)  computer 
for  each  aircraft  OBC  that  is  being  emulated. 
The  internal  structure  of  the  GP  computer  is 
changed  through  microcode  to  allow  It  to 
accept  the  instruction  set  of  the  OBC.  This 
requires  a detailed  knowledge  of  the  internal 
workings  of  the  aircraft  computer  that  is 
being  emulated,  to  insure  that  the  emulator 
internally  looks  exactly  like  the  OBC.  With 
this  approach,  the  actual  aircraft  OFP  is 
directly  loaded  into  the  emulator  and  the  OFP 
"thinks"  it  is  executing  in  the  actual 
aircraft  computer. 

Advantages  of  a Hardware  Emulator: 

Because  the  emulator  uses  a GP  computer,  it 
is  much  easier  to  integrate  into  the  simu- 
lator than  the  actual  OBC.  Interface 
problems  are  minimized  because  of  commercially 
available  peripherals.  This  approach  has  the 
same  advantages  as  the  OBC  approach  of  rapid 
OFP  update  and  simulator  exercise  of  OFP 
software,  while  the  hardware  cost  is  less 
because  a less  expensive  GP  computer  is  used. 


Disadvantages  of  a Hardware  Emulator: 
The  emulator  has  all  the  disadvantages  of 
the  aircraft  hardware  approach,  except  as 
stated  above.  Additionally,  the  GP  computer 
must  be  of  higher  performance  than  the  OBC 
it  emulates,  and  a detailed  description  of 
its  internal  architecture  must  be  available. 


Software  Translator 

In  this  approach  a computer  program  is 
designed  to  accept  as  input  a set  of  OFP 
instructions  and  provide  as  output  a set  of 
computer  program  instructions  which,  when 
executed  in  the  simulator  computer, 
accomplish  the  "same"  performance  as  the 
actual  aircraft  OFP.  The  translation  process 
is  usually  accomplished  off-line  and  may 
require  several  passes  before  an  executable 
code  is  generated.  The  translator  can  use 
the  aircraft  programs  and  media,  but  must 
operate  on  the  OFP  before  it  can  be  used  in 
the  simulator. 

Advantages  of  a Software  Translator: 
Development  costs  are  nonrecurring  and  hard- 
ware costs  are  minimized.  Aircraft  OFP  and 
media  can  be  used  and  updates  can  be 
accomplished  relatively  quickly. 

Disadvantages  of  a Software  Translator: 
High  level  organic  support  capability  may  be 
required  to  resolve  aircraft  OFP  coding  not 
recognized  by  the  translator,  including 
programmer  tricks  which  take  advantage  of  the 
hardware  for  which  the  OFP  was  written. 

There  is  no  guarantee  that  every  tape  change 
will  work  in  the  simulator  since  verification 
of  the  translator  is  very  difficult. 
Development  of  the  translator  must  be 
considered  a high  risk  item,  both  in  the 
technical  and  cost  area. 

Cross  Compiler 

In  the  past,  weight  and  time  constraints 
had  made  it  necessary  for  the  OFP  to  be 
written  in  assembly  language.  This  made  the 
OFP  very  efficient  as  far  as  core  and  CPU 
time  was  concerned,  but  it  was  hard  to  read 
and  to  modify.  With  the  advance  of  hardware 
technology,  it  became  feasible  to  write  the 
OFP  in  High  Order  Languages  (HOL),  such  as 
Fortran  or  Jovial,  which  are  easier  to  read 
and  allow  easier  code  modification.  The 
usual  procedure  is  to  compile  and  assemble  the 
HOL  program  on  a large  ground-based  computer 
(such  as  an  IBM  370)  and  execute  the 
resultant  machine  code  in  the  aircraft 
computer.  It  would  be  feasible  to  design 
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another  cross  compiler  which  would  take  the 
same  HOL  program  and  process  it  to  obtain  a 
program  which  can  be  executed  in  the 
simulator  computer.  Thus  the  aircraft  OFP 
would  be  executed  in  the  simulator  without 
the  use  of  aircraft  hardware. 

Advantages  of  a Cross  Compiler:  Develop- 

ment costs  and  technical  risks  are  minimized 
because  of  extensive  industry  experience  in 
compiler  development.  The  aircraft  OFP  in 
HOL  format  can  be  used  (allowing  exercise  of 
aircraft  software  in  the  simulator)  and 
relatively  quick  updates  of  OFP  changes  can 
be  accomplished. 

Disadvantages  of  a Cross  Compiler:  If 

the  OFP  is  not  100%  in  HOL  and/or  changes 
are  made  after  the  compilation  process  and 
before  the  OFP  is  finalized,  changes  may 
still  have  to  be  made  to  the  simulator  OFP 
to  obtain  the  same  effect  as  in  the  aircraft. 
This  would  require  a software  maintenance 
team;  however,  as  with  the  functional  simu- 
lation approach,  such  a team  is  required  for 
simulator  software  support  anyway,  and  the  use 
of  a cross  compiler  would  give  the  team  a 
better  handle  on  the  problem  than  would,  for 
instance,  a translator  because  of  the 
greater  visibility  of  HOL  over  assembly  code. 

A problem  may  still  arise  with  this 
approach  in  case  of  programming  tricks 
(although  they  are  less  likely  to  be  found  in 
HOL  than  in  assembly  language  programming). 
Another  possible  problem  may  arise  in 
implementing  training  requirements  such  as 
freeze  and  reset;  however,  these  could 
probably  be  implemented  by  additional  soft- 
ware modules.  If  the  OFP  uses  internal 
clocks,  the  timing  of  which  are  system 
dependent,  the  program  may  not  execute  exact- 
ly the  same  as  in  the  aircraft;  however,  this 
is  a poor  programming  practice  and  unlikely 
to  occur. 

Functional  Simulation 

A functional  simulation  of  the  OBC 
functions  consists  of  computer  programs 
developed  from  the  OFP  documentation.  These 
programs  could  be  designed  as  separate 
modules,  communicating  through  a common  data 
base,  such  that  no  special  interface  software 
is  required.  Control  and  execution  of  these 
OFP  functional  modules  would  be  handled  in 
the  same  manner  as  the  other  simulation 
modules  and  would  be  designed  and  modularly 
organized  into  the  total  Computer  Program 
System  (CPS).  The  usual  procedure  of 
implementing  a functional  simulation  is  for 
the  simulator  designer  to  analyze  the 
documentation  of  the  aircraft  OFP  and  design 
his  own  software  to  duplicate  those  functions 
which  can  be  observed  by  the  crew  in  the 
simulator  and  are  required  for  the  training 
mission. 


Advantages  of  Functional  Simulation: 

This  approach  can  be  successfully  implemented 
with  low  design  risk  and  also  offers  the 
maximum  flexibility  to  integrate  into  the 
total  system  design.  Technical  risks  are 
low,  due  to  extensive  industry  experience, 
provided  the  OFP  functions  are  well  document- 
ed. Instructional  features,  such  as  freeze, 
reset,  malfunctions,  etc.,  are  readily 
designed  into  the  CPS  and  present  minimum 
difficulties  in  implementation.  Any  future 
instructional  features  are  easily  incorpo- 
rated, because  the  simulator  software  is 
under  complete  control  of  simulator  software 
support  personnel  and  can  be  easily  modified 
to  accommodate  new  requirements.  This 
approach  minimizes  both  development  time  and 
computer  equipment  cost,  since  the  software 
needs  only  to  handle  those  functions  which 
can  be  perceived  and  are  necessary  in  the 
simul ator. 

Disadvantages  of  Functional  Simulation: 
The  accuracy  (i.e.  one-to-one  correspondence 
with  aircraft  functions)  by  which  the  soft- 
ware operates  will  not  be  exactly  the  same 
as  the  actual  OFP.  However,  this  may  not  be 
of  any  consequence,  since  the  simulator  does 
not  require  the  high  fidelity  required  in 
the  aircraft  (since  the  training  task  is  only 
concerned  with  what  the  crew  can  perceive, 
iteration  rates  may  not  need  to  be  as  high  as 
in  the  aircraft,  for  example).  Software 
costs  may  be  high,  since  the  simulator  OFP 
will  have  to  be  developed  from  scratch,  but 
this  may  be  minimized  due  to  the  fact  that 
there  are  many  functions  in  the  aircraft  OFP 
which  are  not  required  in  the  simulator. 
Operational  costs  may  also  be  high,  since  a 
team  of  software  maintenance  personnel  must 
track  all  aircraft  OFP  changes  and  incorpo- 
rate those  changes  which  impact  the  training 
environment.  However,  the  OFP  is  only  a 
small  part  of  the  overall  simulator  CPS  and 
there  must  be  a software  maintenance  team 
anyway  to  provide  organic  support  Another 
possible  disadvantage  of  this  approach  is 
that  the  simulator  may  lag  behind  the  air- 
craft in  OFP  updates.  Certainly,  instances  of 
this  occurring  can  be  pointed  out,  but  the 
problem  is  more  a management  than  an 
engineering  one.  It  can  be  resolved  if  the 
simulator  team  is  kept  informed  of  future  OFP 
changes,  so  that  they  can  work  on  those 
changes  which  impact  the  simulator  concurrent- 
ly with  the  aircraft  OFP  team.  Control  of  the 
OFP  configuration  is  maintained  by  means  of  a 
Configuration  Control  Board  (CCB)  which  must 
approve  all  proposed  OFP  changes.  Notifica- 
tion of  all  CCB  approved  changes  would  allow 
the  simulator  support  team  adequate  time  to 
incorporate  these  changes.  There  should  be 
no  reason  then  for  the  simulator  to  lag 
behind  the  aircraft. 
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SIMULATION  OF  B-l  ON-BOARD  SYSTEMS 

Devices  using  ROMs  or  PROMs  can  be  func- 
tionally simulated,  emulated,  or  the  actual 
hardware  may  be  used.  Because  the  program 
resides  in  hardware,  neither  translator  nor 
cross  compiler  are  viable  options.  With  com- 
puters using  core  memory,  all  of  the  above 
options  may  be  viable.  Most  of  the  processors 
used  in  the  B-l  use  PROMs.  It  was  found  that 
all  of  these  devices  could  be  best  simulated 
functionally,  either  because  they  monitored 
real-world  environment  (such  as  the  air  data 
computers  or  the  engine  instruments  signal 
converters)  or  because  their  function  could  be 
implemented  in  software  in  a more  cost  effec- 
tive manner  (such  as  the  electrical  multiplex 
processors,  which  operates  on  sets  of  Boolean 
logic  equations).  Table  1 lists  several  B-l 
devices,  their  function,  and  the  reasons  for 
functional  simulation.  In  addition,  four 
major  systems  utilizing  programmable  digital 
processors  are  discussed  in  greater  detail  in 
the  following  paragraphs. 

Electrical  Multiplex  System  (EMUX) 

This  system  provides  the  control  of 


electrical  power  in  the  aircraft  and  provides 
the  means  of  transferring  data  between  sub- 
systems. The  B-l  has  two  EMUX  systems.  The 
forward  avionics  bay  and  the  forward  and  aft 
weapons  bays  are  serviced  only  by  the  left 
EMUX,  while  the  central  weapons  bay  and  aft 
avionics  radar  bay  are  serviced  only  by  the 
right  EMUX.  The  crew  compartment,  the  central 
avionics  bay,  and  the  wheel  well  equipment  bay 
are  serviced  by  both  EMUX  systems.  Each 
EMUX  system  has  two  control  boxes  (one  being 
primary  and  the  other  backup]  which  control 
message  traffic  and  access  data,  and  provide 
logic  processing,  format  and  distribute  out- 
puts. In  addition,  each  side  has  a Central 
Integrated  Test  System  (CITS)  interface  box. 
EMUX  samples  the  status  of  switches,  circuit 
breakers,  and  power  controllers  in  the  air- 
craft. It  operates  on  a set  of  Boolean  logic 
equations  and  issues  power  controller 
comnands  if  the  equation's  logical  value  is 
true.  In  addition  to  data  transfers,  EMUX 
also  does  serial  to  discrete  and  discrete  to 
serial  data  conversion;  conducts  self-tests; 
outputs  data  to  CITS  and  accepts  test 
comnands  from  CITS  in  the  ground  mode.  The 
structure  of  EMUX  makes  it  convenient  to 


TABLE  1 

SUMMARY  OF  B-l  DIGITAL  DEVICES 

Device 

Function 

Reasons  for  Using 
Functional  Simulation 

Central  Air  Data 
Computer  (CADC) 

Uses  air  pressure  & temperature 
to  calculate  true  airspeed,  mach 
number,  and  related  data. 

It  is  mainly  an  A-to-D  converter  and 
is  stable  in  design  and  not  likely 
to  change. 

Rotation-Go-Around/ 

Angle-of-Attack/ 

Air  Vehicle  Limit 
(RGA/AOA/AVL) 

Provides  standardized  AOA,  air 
vehicle  limits,  and  pitch  steer- 
ing information  to  cockpit 
instruments. 

It  is  mainly  an  interface  for  cockpit 
instruments  and  is  stable  in  design 
and  not  likely  to  change. 

Fuel  and  Center  of 
Gravity  Management 
System  (FCGMS) 

Automatically  maintains  the  CG 
by  transferring  fuel  between 
tanks. 

This  device  can  be  implemented  with 
simpler  algorithms  in  the  simulator 
computer  to  provide  crew  training  in 
its  operation. 

Gyro  Stabilization 
System  (GSS) 

Provides  heading,  pitch,  roll 
attitude,  and  rate-of-turn 
reference  data. 

The  simulator  "moves"  in  a simulated 
environment,  hence  there  is  nothing 
for  the  GSS  to  sense.  Its  output 
should  be  functionally  simulated. 

Engine  Instruments 
Signal  Conditioning  & 
Distribution  Unit  (SCDU) 

Processes  engine  sensor  data  for 
the  cockpit  engine  instruments. 

It  is  an  A-to-D  converter  for  the 
flight  instruments  and  is  stable  in 
design  and  not  likely  to  change. 

FI ight  Instruments 
Signal  Converter  (FISC) 

A-to-D  and  D-to-A  conversion 
to  interface  the  ACUC  with 
various  aircraft  systems. 

Functional  simulation  of  this 
interface  is  simplier  and  less 
costly. 

Vertical  Situation 
Display  (VSD) 

Generates  the  real-time  video 
display  of  flight  parameters 
such  as  roll  and  pitch 
attitude,  AOA,  airspeed 
and  altitude. 

In  a large  simulator  buy,  it  is 
more  cost-effective  to  functionally 
simulate  its  functions  and  use 
non-aircraft  parts  for  the  display. 

implement  in  the  simulator.  In  the  past, 
simulator  engineers  in  designing  the  simula- 
tor had  to  search  documentation  to  ascertain 
what  effect  each  and  every  switch  had  on  the 
characteristics  of  the  aircraft,  i.e.,  what 
events  would  the  crew  perceive  if  a switch 
were  thrown,  a button  were  pushed,  etc.?  The 
EMUX  documentation  provides  the  logic 
equations  in  a very  simple  format.  Implemen- 
tation of  these  functions  should  be  simpli- 
fied over  previous  simulator  developments. 

It  is  not,  however,  necessary  to  physically 
include  the  EMUX  system,  since  the  simulator 
has  no  Station  Logic  Units  (SLU),  such  as 
bombbay  doors  or  launcher  racks,  to  control, 
and  data  transfer  will  consist  of  core-to- 
core  communication  within  the  simulator  host 
computer.  Thus  , only  the  functions  of  EMUX 
should  be  simulated. 

Central  Integrated  Test  System  (CITS) 

The  CITS  function  is  to  detect  faults 
and  isolate  faults  to  a unit,  and  display 
the  fault  detection/isolation  information  to 
the  crew.  It  also  records  this  information 
on  a printer  and  maintenance  recorder  for 
post-mission  debriefing  and  maintenance 
purposes  (neither  is  accessible  in  flight). 

The  CITS  collects  information  via  Data 
Acquisition  Units  (DAU),  and  interfaces  with 
EMUX  and  the  avionics  complex.  Crew  infor- 
mation is  displayed  on  the  CITS  control  and 
display  panel  (CCD)  which  is  situated  in  the 
aft  station  between  the  two  operators.  Upon 
a detected  failure,  CITS  will  transmit  a 
warning  message  to  be  displayed  on  the  CCD's 
Alphanumeric  Display  (AND)  and  turn  on  the 
light  indicating  which  system  failed.  The 
crew  can  retrieve  any  previous  messages  and 
access  core  memory  of  the  CITS  Dedicated 
Computer  (CDC)  to  obtain  further  information 
on  the  fault.  CITS  communicates  with  its 
peripherals  on  a CITS  multiplex  system  (CMUX). 
In  the  simulator  the  aircraft  malfunctions 
that  are  considered  necessary  for  training 
are  preprogrammed  and  under  control  of  the 
instructor.  Thus  there  is  no  need  to  detect 
faults,  only  to  display  the  indication  of  a 
fault  to  the  crew.  Since  the  printer  and 
recorder  are  not  needed  by  the  crew  in  the 
simulator,  these  CITS  functions  are  not 
necessary.  Likewise,  the  ground  functions 
of  CITS  in  which  it  controls  EMUX  are  not 
needed  since  they  are  maintenance  functions 
only.  The  C&D  function,  including  the 
capability  of  the  crew  to  access  certain 
portions  of  core  in  the  CDC,  can  be  handled 
by  functional  simulation  in  the  simulator 
host  computer. 

Avionics  Control  Unit  Complex  (ACUC) 

The  ACUC  contains  three  SKC  2070 
computers,  two  for  the  offensive  avionics  and 
one  for  the  defensive  avionics.  The  two 
offensive  avionics  computers  operate  in 


parallel,  but  each  can  be  configured  to 
assume  the  functions  of  the  other  in  case  of 
failure  of  one.  Communication  among  the 
computers  is  via  an  avionics  multiplex 
system  (AMUX).  In  addition  to  private  core 
memory,  the  three  processors  share  a mass 
storage  unit  (MSU).  The  MSU  is  a drum  with 
400K  of  32  bit  words  storage  capacity  and 
contains  all  the  computer  programs  and 
mission  data  for  the  avionics  computers.  The 
ACUs  are  loaded  via  the  DEU  or  the  MSU.  The 
Weapon  Delivery  Avionics  Control  Unit 
WDACU)  is  loaded  first.  From  a cold  start 
MSU  not  loaded),  this  is  accomplished  by  the 
0S0  inserting  the  program  tape  into  the  DEU. 
After  the  WDACU  is  loaded,  the  general 
navigation  avionics  control  unit  (GNACU)  can 
be  loaded  either  from  the  DEU  or  from  the 
MSU.  The  defensive  avionics  control  unit 
(DACU)  can  only  be  loaded  from  the  MSU. 

The  Offensive  Flight  Software  (OFS)  is 
executed  in  the  GNACU  and  WDACU.  The  basic 
iteration  rate  is  16,  with  each  62.5  msec 
major  frame  divided  into  four  minor  frames. 
The  functions  are  executed  alternately  during 
each  minor  frame.  In  the  backup  mode  (one 
computer  down)  all  functions  are  executed  in 
the  same  computer.  During  each  minor  frame, 
the  foreground  programs  are  executed  first. 
Any  remaining  time  is  devoted  to  background 
tasks  (Impact  Point  Calculations,  Kalman 
filters,  and  self-tests). 

The  Defensive  Flight  Software  (DFS)  is 
executed  in  the  DACU  in  major  frames  (62.5 
msec)  only.  There  are  presently  no  pro- 
visions to  backup  the  DACU,  although 
technically  it  would  be  feasible  to  have  one 
of  the  offensive  ACUs  operate  in  the  backup 
mode  and  have  the  other  reconfigure  itself 
to  the  DACU.  Implementation  of  this 
capability  would  require  some  engineering 
redesign.  The  OFS  and  DFS  are  programmed 
for  tne  greater  part  (by  contract  at  least 
75%)  in  Jovial  J3B. 

Radio  Frequency  Surveillance/Electronic 
Counter  Measures  System  (RFS/ECMST 

This  system  uses  a pre-processor 
avionics  control  unit  (PACU)  to  process  the 
emitter  data  from  the  antennas  and  control 
the  jammers.  The  PACU  is  loaded  from  the 
MSU  through  the  DACU.  It  contains  the 
Defensive  Order  of  Battle  (DOB)  and  any 
defensive  mission  data  that  has  been  entered 
by  the  DSO  using  the  IKB.  The  PACU  processes 
the  incoming  signals  and  correlates  their 
characteristics  with  the  stored  data  to 
produce  a prioritized  table  of  threats  which 
it  transmits  to  the  DACU.  The  DACU  prepares 
this  data  for  display  on  the  EDUs  and  the 
aural  warning  system  when  appropriate. 

The  PACU  uses  a Litton  LC4516D 
computer.  The  pre-processor  flight  software 
(PFS)  is  presently  coded  in  assembler 
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language,  but  is  in  the  process  of  being  re- 
written in  Jovial  J3B.  A cross  compiler  will 
be  used  to  generate  the  aircraft  OFP.  Since 
the  simulator  will  not  use  actual  antennas 
and  jammers,  the  signal  processing  and 
control  functions  of  the  PACU  are  not  needed 
in  the  simulator.  What  has  to  be  incorpo- 
rated in  the  simulator  are  the  threat 
analysis  and  prioritization  functions.  In 
the  simulator,  the  threat  environment  is 
known  (i.e.,  it  is  a simulated  environment), 
hence  the  functions  of  the  PACU  that  are 
needed  in  the  trainer  should  be  functionally 
simulated.  These  functions  should  include 
the  capability  to  change  threat  and  jammer 
characteristics  {PRF,  power,  etc.)  and  add 
new  threats.  An  additional  option  exists  if 
the  cross-compiler  approach  is  used  to 
simulate  the  ACUC.  In  that  case  it  may  be 
cost  effective  to  design  a cross  compiler 
for  the  PACU  to  incorporate  its  functions 
into  the  simulator.  This  would  minimize  the 
interface  with  the  ACUC. 

ACUC  Simulation 

There  are  several  facts  which  must  be 
taken  into  account  while  considering  the 
simulation  alternatives  for  the  ACUC 
computers : 

1.  The  aircraft  SKC  2070  computer 
costs  about  $600K  per  copy  in  the  nuclear 
hardened  version  and  about  $350K  in  the 
softer  (non-hardened)  version.  It  may  be 
possible  to  acquire  a still  softer  version, 
using  non-mil  spec  parts,  in  the  $1 00- 1 50K 
range.  The  problem  with  the  last  approach 
is  that  we  are  essentially  dealing  with  a 
new  computer,  whose  performance,  reliability, 
etc.  is  unknown.  Additionally,  one  advantage 
of  using  aircraft  hardware  (i.e.,  reduced 
logistics  problems)  would  be  lost  if  non- 
hardened  versions  are  used.  Even  the  lowest 
cost  version  would  still  cost  S300-450K  per 
simulator  for  all  three  computers.  It  may  be 
possible  to  eliminate  one  computer  by 
operating  one  ACU  in  backup  mode  permanently, 
and  this  would  eliminate  some  of  the  test 
functions,  but  they  may  not  be  necessary  to 
meet  the  training  requirements  anyway.  Even 
so,  this  would  eliminate  the  capability  to 
train  crew  members  to  operate  in  both  modes. 

Implementing  trainer  essential 
functions  are  difficult.  Freeze,  for 
example,  could  be  implemented  simply  by 
initiating  an  interrupt  stopping  the  real- 
time clock  in  the  ACU.  However,  this  would 
cause  all  displays  to  go  dead,  unless  the 
information  necessary  for  display  is  somehow 
retained  outside  the  ACU.  To  implement 
parameter  freeze  requires  modification  of  the 
OFP  for  simulator  use.  This  means  that 
either  the  simulator  OFP  will  not  be  identical 
to  the  aircraft  OFP,  or  the  aircraft  will  have 
to  fly  with  simulator  peculiar  functions  in 


the  OFP.  Other  problems  include  record  and 
playback  capability,  auto  demonstrations, 
crash  override,  and  malfunction  insertion 
and  control . 

2.  Data  concerning  the  internal 
operations  of  the  SKC  2070  computer  are 
company  proprietary.  This  will  make  it 
difficult  to  design  an  emulator  using  a 
microprogrammable  GP  computer  which  reflects 
the  structure  of  the  OBC  exactly.  There  is 
therefore  no  guarantee  that  a change  in  the 
aircraft  OFP  will  have  the  same  effect  in 
the  simulator  and  some  simulator  peculiar 
OFP  support  activity  may  be  recuired  with 
each  tape  release.  As  with  the  actual 
hardware  approach,  the  same  situation  exists 
concerning  simulator  peculiar  functions 
with  the  same  problems  of  having  either  two 
different  OFPs  (one  for  the  aircraft  and  one 
for  the  simulator)  or  having  simulator 
peculiar  functions  in  the  aircraft  OFP. 

If  this  approach  is  used,  it 
would  be  very  cost  effective  to  use  the 
same  microprogrammable  GP  computer  for  the 
simulator  host  computer(s)  to  allow  common- 
ality of  equipment  and  ease  the  maintenance 
function.  This  would  limit  the  choice  of 
simulator  computers  to  32  bit  microprogram- 
mable GP  processors.  An  indepth  study  of  the 
SKC  2070  and  available  GP  computers  will  be 
required  to  determine  which  computers  are 
likely  candidates  for  an  emulator  approach. 

3.  The  present  method  of  preparing 
the  aircraft  program  tape  requires  the  use  of 
a cross  compiler  (this  is  also  true  of  the 
pre-processor  flight  software,  which  was 
covered  in  the  previous  section).  The  design 
and  cost  risks  of  a cross  compiler  to 
accommodate  the  simulator  computer  would  be 
low.  The  software  is  organized  such  that  all 
programs  are  either  in  Jovial  J3B  or  in 
assembler  language  (no  in-line  coding  is 
allowed).  This  would  make  it  easier  to 
accomplish  changes  to  the  program.  Simulator 
functions  peculiar  to  the  training  environ- 
ment could  be  inserted  as  additional  code  and 
new  requirements  could  be  implemented  without 
modification  of  the  aircraft  OFP.  The  update 
capability  is  not  as  good  as  with  the 
previous  two  approaches,  since  with  each  new 
tape  release  at  least  some  analysis  and 
possibly  recoding  will  have  to  be  done  to 
obtain  a simulator  OFP.  However,  this 
approach  gives  much  greater  visibility  of  the 
total  system  and  any  bugs  can  be  fixed  more 
easily  than  with  the  hardware  or  emulator 
approach. 

4.  The  translative  approach  has  all 
the  disadvantages  of  the  cross-compiler 
approach,  I.e. • the  translator  requires  off- 
line processing  to  produce  a simulator  OFP, 
including  engineering  analysis  and  computer 
run  time.  In  addition,  any  code  change  would 
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have  to  be  done  at  the  assembler  or  machine 
language  level.  This  removes  the  advantage 
of  increased  visibility. 

5.  From  a development  standpoint, 
functional  simulation  of  the  avionics  has  the 
lowest  technical  risk.  This  approach 
minimizes  the  total  software  and  hardware, 
since  the  designer  can  eliminate  all  functions 
which  are  necessary  in  the  aircraft,  but  have 
no  equivalent  function  in  the  simulator. 

This  is  particularly  true  in  the  B-l  where 
much  of  the  software  is  concerned  with 
testing,  statistical  computations  (Kalman 
filters),  and  control  and  display  (C&D)  - all 
of  which  are  either  unnecessary  in  the 
simulator  or,  as  in  the  case  of  CAD,  can  be 
done  more  efficiently  iri  a GP  computer.  This 
approach  will  require  that  OFP  change  speci- 
fications be  made  available  as  early  as 
possible  to  allow  simulator  software  support 
teams  to  keep  the  simulator  OFP  up-to-date 
with  the  aircraft. 

6.  There  may  be  other  approaches, 
including  combinations  of  the  above;  however, 
they  have  not  been  proven  in  the  training 
simulator  environment. 

CONCLUSIONS 

All  analog  computers  and  all  digital 
processors,  with  the  exception  of  the  three 
com:  jters  in  the  ACUC  (and  possibly  the  PACU, 
if  the  cross  compiler  approach  is  used)  should 
be  functionally  simulated.  Two  approaches  to 
implementing  the  ACUC  computers  are  considered 
more  cost  effective  over  the  life-cycle  of  the 
B-l  simulator  than  the  other  approaches.  The 
approach  which  presently  appears  to  be  the 
better  of  the  two  is  functional  simulation. 

The  following  reasons  are  given  in  support  of 
this  conclusion: 

1.  Low  risk.  This  approach  has 
been  implemented  on  many  training  simulators 
and  it  is  a proven  and  reliable  approach. 

Z.  Short  development  time.  Be- 
cause of  extensive  industry  experience  with 
this  approach,  development  time  will  be  mini- 
mized. There  are  no  unknown  areas  or 
surprises  which  can  delay  the  program.  This 
is  particularly  important  in  the  case  of  the 
B-l  simulator  which  has  a severe  time 
constraint. 


interface  with  the  simulator,  contractor 
overhead  charges  for  the  hardware  procure- 
ment, and  the  software  cost  of  having  to 
implement  a more  rigorous  environment  model 
if  the  OBC  is  used. 

4.  High  flexibility.  Simulator 
peculiar  requirements  such  as  freeze,  reset, 
malfunctions , and  parameter  freeze  are  easily 
implemented.  Any  future  requirements  can  be 
easily  implemented.  At  the  same  time,  the 
simulator  OFP  is  not  hardware  dependent,  i.e., 
if  the  aircraft  computer  is  replaced  or  modi- 
fied, the  simulator  OFP  need  only  be  modified 
if  there  is  a change  in  performance  that  the 
crew  can  perceive.  In  such  a case  there 
would  be  no  high  hardware  costs  associated 
with  the  simulator. 

5.  Control  of  configuration  is 
maintained  by  the  simulator  community.  This 
may  be  the  most  important  advantage  of 
functional  simulation,  i.e.,  the  simulator  is 
not  dependent  upon  the  aircraft  OFP  for  its 
training  capabilities.  This  would  not  be  the 
case  if  the  actual  hardware  or  an  emulator 
were  to  be  used,  since  the  aircraft  OFP 
would  then  have  to  contain  the  freeze,  reset, 
etc.,  capabilities  which  are  necessary  for  the 
simulator.  In  other  words,  the  simulator 
requirements  would  be  in  direct  competition 
with  the  operational  requirements  for  time 
and  core  in  the  OBC,  and  dependent  on  the 
aircraft  community  for  implementation. 

6.  Good  update  capability. 
Traditionally  this  approach  has  been  accused 
of  causing  long  delays  in  simulator  updates 
and  expensive  ECPs  to  keep  the  simulator 
current  with  the  aircraft.  These  facts  may 
have  been  true  in  the  past,  but  are  not 
necessarily  true  today.  The  Tactical  Air 
Command  (TAC)  has  had  great  success  with 
their  Development  Technician  Team  (DTT)  to 
update  all  simulator  software,  including  the 
OFP  portions.  Last  year  TAC  rejected  a 
suggestion  to  incorporate  the  TC-2  computer 
into  the  A-7D  simulator  on  the  grounds  that 
they  had  no  update  problems.  They  felt  that 
as  long  as  their  DTT  was  kept  informed  of 
future  OFP  changes,  they  could  beat  the  air- 
craft in  implementing  a change.  They  also 
felt  that  the  approach  was  too  expensive, 
degraded  their  training  capability,  and 
would  take  too  long  to  implement.  HQ  USAF/XO 
and  AFSC  concurred  with  TAC's  position. 


3.  Low  cost.  The  cost  to  rewrite 
the  OFP  functions  for  the  simulator  is  non- 
recurring and  amounts  to  less  than  1%  of  the 
projected  total  B-l  simulator  cost.  Even 
assuming  that  soft  versions  of  the  OBC  at  a 
reduced  cost  can  be  obtained,  the  hardware 
cost  will  still  be  several  times  higher 
than  the  cost  of  a functional  simulation. 

This  does  not  take  into  account  the  additional 
cost  of  the  AMUX  system,  the  cost  of  the 


The  recommended  method  of  implementing 
OFP  changes  in  the  simulator  is  to  require 
that  the  SAC  equivalent  of  the  DTT  be  kept 
informed  of  any  future  OFP  changes  and  that 
the  funding  for  any  aircraft  ECPs  which 
affect  the  simulator  contain  funding  for 
simulator  ECPs  as  well.  This  will  insure  a 
timely  update  capability  for  the  simulator 
OFP  without  rg^u|rin^  a |iign  post/high  risk 


approach  to 


ating  the  avionics  system. 
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The  other  viable  approach  is  the  cross 
compiler.  Its  cost  is  comparable  to  that  of 
the  functional  approach  and  is  much  less  than 
that  of  aircraft  OBCs  or  emulators.  It  has 
a slightly  higher  development  risk  than 
functional  simulation,  since  it  has  not  been 
proven  in  a training  simulator,  but  it  would 
give  maintenance  personnel  a better  chance 
of  tracking  any  bugs  in  the  system  which  may 
be  impossible  to  find  with  the  aircraft  OBC 
or  emulator  approaches.  It  allows  some  of 
the  functional  simulation  options  of  deleting 
code  which  has  no  impact  in  the  simulator  or 
adding  subroutines  to  take  care  of  freeze, 
reset,  etc.  It  would  also  allow  the  option 
of  using  Jovial  J3B  as  the  simulator 
computer  language  to  minimize  interface 
problems. 

To  take  care  of  the  non-Jovial  code 
(the  executive,  I/O,  etc.),  the  simulator 
contractor  could  write  new  code,  or  else 
microprogram  the  simulator  computer  to  accept 
the  instruction  set  of  the  SKC  2070.  This 
approach  is  not  as  technically  proven  as  the 
functional  approach,  but  it  is  the  best  of 
the  approaches  which  use  the  aircraft  OFP, 
and  thus  would  be  the  best  approach  if 
exercising  the  aircraft  software  in  the 
simulator  is  required.  This  does  not  mean 


that  the  other,  approaches  are  without  merit. 
On  the  contrary,  with  different  applications 
and  requirements,  the  emulator,  for  example, 
may  be  the  best  approach.  But  for  the  B-I 
training  simulator  program,  those  approaches 
are  not  recommended  for  the  reasons  stated 
above. 

A further  trade-off  study  should  be 
made  to  determine  which  of  these  two 
approaches  is  best  suited  for  the  B-l 
simulator  in  terms  of  life-cycle  cost  and 
training  effectiveness.  It  should  be  noted 
that  the  aircraft  aft  station  is  behind  in 
development  compared  to  the  front  station, 
and  hence  a delay  of  one  year  in  production 
of  the  simulator  aft  station  would  prevent 
costly  ECPs  to  keep  the  simulator 
configuration  current  with  the  aircraft. 

This  time  period  can  be  used  to  accomplish 
the  recommended  study.  The  study  should, 
as  a minimum,  address  the  initial 
development  cost;  the  time  for  development; 
a development  risk  assessment;  an  analysis 
of  anticipated  software  changes  and  their 
impact  on  the  simulator;  the  update 
capability;  the  number  and  qualifications 
of  software  maintenance  personnel  required; 
and  predicted  life-cycle  costs  for  each  of 
the  two  approaches. 
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ABSTRACT 

Ai r-to-surface  weapons  delivery  is  one  realm 
of  visual  flight  simulation  which  has  been 
rather  neglected  until  recent  investigations 
were  made  by  the  USAF  to  determine  the 
state-of-the-art  in  this  area.  As  part  of 
this  investigation  (Project  2235,  Air-to- 
Ground  Visual  Eval uation), the  Advanced  Simu- 
lator for  Undergraduate  Pilot  Training 
(ASUPT)  system  was  expanded  to  include  the 
additional  capabilities  required  to  perform 
ai r-to-surface  weapons  delivery.  Evalua- 
tions of  the  various  systems  under  consider- 
ation have  shown  the  ASUPT  computer  image 
generation  approach  to  air-to-surface  visual 
simulation  to  be  the  most  viable.  This 
paper  summarizes  the  engineering  modifica- 
tions made  to  the  ASUPT  system  for  Project 
2235  and  presents  the  operational  capabili- 
ties of  the  new  system  configuration. 

INTRODUCTION 

Project  2235  arose  from  an  urgent  require- 
ment to  determine  the  state-of-the-art  in 
visual  air-to-surface  (A/S)  mission  simula- 
tion. Specifically,  the  project  supports 
the  establishment  of  Required  Operational 
Capabilities  (ROC's)  for  the  proposed  A-10 
and  F- 76  Mission  Simulators.  Air  Staff  RDQ 
and  the  Aeronautical  Systems  Division/Sim- 
ulator Systems  Program  Office  (ASD/SIMSPO/ 
SD24F)  of  Air  Force  Systems  Command  (AFSC) 
felt  that  a study  to  assess  visual  simula- 
tion capabilities  was  necessary  to  reduce 
the  performance  and  cost  risks  of  procuring 
aircrew  simulators  which  require  A/S  capa- 
bilities. A consortium  of  the  Air  Staff, 
ASD/SIMSPO  and  AFSC  arrived  at  a decision, 
dictated  to  a large  extent  by  available 
funds  and  time,  to  make  the  assessment  by 


evaluating  three  systems  currently  in  the 
field.  These  systems  are  the  Advanced  Simu- 
lator for  Undergraduate  Pilot  Training  (ASUPT) 
System  located  at  Air  Force  Human  Resources 
Laboratory,  Flying  Training  Division  (AFHRL/ 
FT),  Williams  AFB,  Arizona;  the  Large  Ampli- 
tude Multi -Mode  Aerospace  Research  Simulator 
(LAMARS)  located  at  the  Flight  Dynamics 
Laboratory  (FDL),  Wright  Patterson  AFB,  OH; 
and  the  Simulator  for  Air-to-Air  Combat 
(SAAC)  F-4E  No.  18  combined  systems  located 
at  Luke  AFB,  AZ.  Each  of  these  systems 
utilized  a different  approach  to  visual  simu- 
lation: ASUPT  features  a Computer  Image  Gen- 

eration (CIG),  wide-angle, field-of-view  (FOV) 
cathode  ray  tube  (CRT)  display;  LAMARS,  an 
area-of-interest  ( AO  I ) dome  projection  of  a 
model  board;  and  the  SAAC,  wi de-angle, FOV 
CRT  display  with  a cameral  model  AO  I of  the 
F-4E  terrain  model  board. 

AFHRL/FT  became  actively  involved  in  Project 
2235  on  8 July  1975  at  the  Test  and  Support 
Planning  Meeting  held  at  Wright  Patterson 
AFB.  A project  plan  was  developed  at  this 
meeting  to  serve  as  a basis  for  all  agree- 
ments among  project  participants  and  out- 
lining the  specific  A/S  task  objectives  to  be 
evaluated.  Shortly  thereafter,  AFHRL/FTE 
developed  an  Engineering  Development  Plan  to 
define  the  scope  of  ASUPT's  ability  to 
respond  to  Project  2235 ' s objectives,  to 
establish  functional  task  areas  requiring 
engineering  development,  and  to  project  a 
schedule  for  the  ASUPT  engineering  develop- 
ment. 

It  was  confirmed  that  ASUPT  could  provide  the 
capability  to  perform  all  of  the  A/S  mission 
requirements  established  in  the  overall 
Project  2235  Test  and  Evaluation  Plan; 
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however,  additional  development  was  required 
in  both  hardware  and  software  areas. 

The  purpose  of  this  paper  is  to  describe  the 
expansion  of  the  ASUPT  system  features  ter  in- 
clude A/S  mission  simulation  and  to  report 
the  operational  results  thereof. 

General  System  Description.  The  ASUPT  system 
consists  of  two  T-37B  simulator  cockpits. 

Each  cockpit  with  G-seat  is  mounted  on  a 
six-degree-of-freedom  synergistic  motion 
base  surrounded  by  seven  cathode  ray  tubes 
(CRT's)  with  special  infinity  optics  provid- 
ing a wide-angle, field-of-view  (FOV)  visual 
display.  The  visual  scene  is  produced  by 
means  of  a computer  image  generation  (CIG) 
technique  which  provides  a perspective  two- 
dimensional  image  of  an  environmental  model 
defined  in  three  dimensional  vector  space 
and  stored  in  computer  memory.  Unique  char- 
acteristics of  this  visual  system  are: 

1.  Wide-angle, field-of-view 

2.  Unrestricted  viewpoint  position  and 
attitude 

3.  Unlimited  number  of  environmental  data 
bases  which  can  be  modified,  amended  and 
constructed  with  reasonable  effort  and  little 
expense 

4.  Large  gaming  area  (1250nm  X 1250nm) 

5.  Provision  for  an  independent  moving 
model  which  may  take  the  form  of  lead/FAC 
aircraft,  moving  ground  target,  SAM,  etc. 


Operational  A/S  Requirements  Provided  by  the 
Original  System  Configuration-)  The  ASUPT 
system,  designed  to  accommodate  research  in 
undergraduate  pilot  training,  easily 
facilitates  those  Project  2235  scenarios  en- 
compassing UPT  missions.  Tasks  such  as 
takeoffs,  departures,  approaches,  landings, 
aerobatics  and  formation  flight  during  day, 
dusk,  night  and  varying  ceiling  and  visi- 
bility conditions  are  readily  accomplished. 


Figure  1.  Advanced  Instructor/Operator  Station 


optical  gunsight  for  an  A-37  aircraft  was  pro- 
cured, modified,  and  installed  in  one  of  the 
two  cockpits.  The  stick  triggers  were  wired, 
activated  and  integrated  with  the  simulator 
computer.  Algorithms  were  developed  to  pro- 
vide paths  for  the  ordnance,  surface-to-air 
missile  (SAM),  and  moving  ground  target. 
Significant  software  was  developed  to  provide 
visual  display  of  ordnance  ground  impacts 
and  the  two  graphics  displays  were  reprogram- 
med to,  provide  for  delivery  scoring.  Also, 
missions  utilizing  the  FAC/lead  aircraft  re- 
quired a moving  model  visible  at  ranges  far 
in  excess  of  those  for  formation  flight,  again 
necessitating  software  program  modifications. 
To  handle  the  extra  software  in  an  efficient 
manner,  8K  of  computer  core  memory  and  a 24 
million  byte  moving-head  disk  drive  were  pur- 
chased and  installed. 


The  observance  and  monitoring  of  pilot  per- 
formance and  aircraft  maneuvers  is  also 
provided.  At  the  various  instructor-operator 
stations,  one  may  survey  the,  situation  by 
means  of  duplicate  instruments,  closed  cir- 
cuit TV,  CRT  monitors,  and  graphics  displays. 
(See  Fig.  1 AIOS) 

Operational  A/S  Requirements  Necessitating 
Additional  Development.  In  addition  to  the 
requirements  imposed  by  A/S  scenarios  with 
tasks  common  to  those  in  UPT,  are  the  ones 
more  directly  related  to  weapons  delivery. 
These  requirements  necessitated  the  acquisi- 
tion, modification  and  development  of  both 
hardware  and  software.  First,  an  entirely 
new  visual  data  base  was  designed  and  de- 
veloped to  provide  an  appropriate  environment 
for  ai r-to-surface  weapons  delivery.  A CA503 


Operational  Capabilities.  The  environmental 
gaming  area  is  a square  area  36  nautical  miles 
on  a side  and  consists  of  random  surface  tex- 
ture, an  airfield,  a gunnery  range,  and  two 
tactical  complexes.  (See  Fig.  2) 

The  airfield  (see  Figs.  3 and  4)  is  composed 
of  a 12,000  foot  fully-marked  and  lighted 
runway  with  numerous  three-dimensional  fea- 
tures to  provide  the  necessary  velocity  and 
altitude  cues. 

The  conventional  gunnery  range  is  in  a base- 
ball diamond  pattern  as  shown  in  Fiq.  5 and 
consists  of  two  bomb  circles,  two  r o bomb 
targets,  and  two  strafe  targets.  Bui  dings 
scattered  throughout  the  area,  range  towers, 
and  as  shown  in  Fig.  6,  trucks  in  the  bonti 
circles  provide  the  three-dimensional  cues. 
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Figure  2.  Environmental  Gaming  Area 


Figure  4.  Threshold 


Figure  3.  Airfield  at  Dusk 


Figure  5.  Gunnery  Range 


One  of  the  tactical  ranges  consists  of  two 
small  towns  about  four  miles  apart  with  a 
surface-to-air  missile  (SAM)  site  located 
between  them.  As  may  be  seen  in  Fig.  7,  a 
convoy  of  trucks  is  departing  the  northern 
town.  The  other  tactical  area  is  located 
in  a river  valley  between  two  mountain  ranges 
spanned  by  a pair  of  bridges  as  shown  in  Figs. 
8 and  9.  In  the  vicinity  of  one  bridge  is  an 


island  containing  a munitions  manufacturing 
facility.  Contained  in  the  munitions  complex 
are  a factory,  ammo  dump,  watch  towers, 
barracks,  warehouses,  dock,  ship,  jeeps, 
trucks,  ambulance,  tanks,  antiaircraft 
artillery  emplacements,  helipad  (see  Fig.  10) 
and  helicopter.  The  mobile  features  in  this 
area  change  their  locations  during  the  day, 
dusk,  and  night  envi ronments . 
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Figure  7.  Truck  Convoy 


Within  the  environment,  the  pilot  may  "fire" 
the  cannon  and  "drop"  bombs  at  will.  Visual 
ground  impacts  are  displayed  for  both  strafe 
and  bonto,  and  when  a three-dimensional  fea- 
ture is  hit,  it  is  deleted  from  the  display 
scene.  This  may  be  observed  in  Fig.  11, 
depicting  a circular  white  impact  near  the 
center  of  the  bomb  circle.  Since  a "hit" 
occurred,  the  truck  has  been  omitted  from  the 
display  scene. 

Two  graphics  displays  have  been  reprooranred 
to  provide  monitoring  and  scoring  of  the  A/S 


Figure  9.  Munitions  Complex 


weapons  delivery  tasks  for  selected  targets 
at  the  advanced  instructor  operator  station 
(AIOS).  These  displays  are  shown  at  the  lower 
left  and  right  portions  of  Fig.  1.  The  Air- 
craft Delivery  Parameter  Display  depicted  in 
Fig.  12  presents  real-time  heading,  altitude, 
airspeed,  dive  angle,  and  G-load  as  digital 
data  at  the  top  edge  of  the  display.  The 
remainder  of  this  display  is  divided  into 
two  parts.  The  top  half  presents  a side  view 
with  fixed  dive  angle  lines  (15°,  30°,  45°, 
60°)  radiating  from  target  and  a dynamic 
aircraft  imaye  moving  from  right  to  left  on 


Figure  10.  Helicopter 


Figure  12.  Aircraft  Delivery  Parameter 
Display 


Figure  11.  Ground  Impact 


the  screen  leaving  a trail  of  dots  in  its  wake 
as  shown  below  the  15°  dive  angle  radial.  The 
bottom  half  is  a top-down  view-oriented  true 
north  also  portraying  a dynamic  aircraft 
image  on  northerly  heading  with  trail  as 
shown.  For  strafe  tasks  this  display  also 
gives  scoring  information  as  a percentage 


of  projectiles  fired  through  the  target  and 
displays  a "FOUL"  if  the  pilot  presses  beyond 
the  range  foul  line.  {See  Fig.  12).  The 
Target-Impact  Display  pictured  in  Fig.  13  is 
a top-down  view  of  the  target  center  oriented 
true  north  with  radial  lines  through  the 
o'clock  positions  and  concentric  circles  at 
150'  and  300'  radius.  Bomb  drop  impacts 
are  indicated  in  their  relative  positions  as 
a circle  with  pass  number.  Heading,  alti- 
tude, airspeed,  G-load,  dive  angle,  and  winds 
at  time  of  release,  in  addition  to  radial 
distance  of  the  impact  from  target  center 
are  accumulated  for  up  to  ten  passes  at.  the 
bottom  of  the  display.  A hardcopy  printout 
of  this  information  is  also  available. 

With  both  cockpits  "flying"  independently, 
each  viewing  the  other  as  the  moving  model, 
formation,  mutual  support  and  forward  air 
control  mission  may  be  performed.  An  image 
size  control  program  was  implemented  to 
introduce  a deadband  range  for  which  the 
moving  model  perspective  image  remains 
constant.  Upon  reaching  the  limits  of  this 
range,  the  image's  size  will  again  commence 
perspective  changes.  This  was  done  in  an 
attempt  to  alleviate  the  resolution  problem 
occurring  with  the  aircraft  at  ranges  in 
excess  of  2500  feet.  Fin.  14  shows  the  mov- 
ing model  of  the  T-37  aircraft  which  is  used 
for  formation  flight.  A more  simplistic 
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Figure  13.  Target  Impact  Display 


model  is  used  for  the  lead  and  forward  air 
control  aircraft.  Indication  of  a target  by 
a forward  air  control  aircraft  is  simulated 
by  "smoke"  marking  of  the  convoy  which  may 
be  turned  on  and  off  at  the  operator 
station  (see  Fig.  7). 


(See  Fig.  15)  and  will  track  the  aircraft.  In 
Fig.  16,  the  launch  of  the  SAM  is  observed 
through  the  gunsight  in  the  simulator  cockpit. 
The  missile  may  be  averted  if  proper  evasive 
tactics  are  performed  in  response  to  the 
missile's  trajectory;  otherwise  a "kill"  is 
experienced  which  results  in  a simulator 
crash  condition. 


Figure  15.  SAM  Site 


Figure  14.  Formation  Aircraft 


Ground  Fire  is  simulated  by  muzzle  flashes  of 

the  antiaircraft  artillery  (see  Fig.  15) 
and  tanks.  Aerial  flak  bursts  also  accom- 
pany the  antiaircraft  artillery  in  the 
vicinity  of  the  munitions  complex.  A missile 
may  be  repeatedly  launched  from  the  SAM  site 


Figure  16.  SAM  Launch 
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Patrolling  the  SAM  site  and  the  highway  be- 
tween the  two  towns  is  a tank.  When  attacked 
and  experiencing  a "near  miss,"  the  tank  will 
leave  the  road,  randomly  changing  speed  and 
direction  to  confuse  the  attacker.  Follow- 
ing an  appropriate  interval,  the  tank  will 
resume  its  normal  patrol. 

CONCLUSION 

In  February  1976  an  evaluation  of  the  system 
was  performed  by  six  Tactical  Air  Connand 
fighter  pilots  highly  experienced  in  air-to- 
surface  weapons  delivery.  Each  pilot  "flew" 
ten  mission  scenarios  of  approximately  one 
hour  each.  These  scenarios  were  representa- 
tive of  all  phases  of  A/S  missions  including 
takeoffs,  landings,  traffic  patterns,  aero- 
batics, formation,  low  level  navigation,  con- 
ventional range  operations  and  deliveries 
and  tactical  operations  and  deliveries.  The 
range  deliveries  included  10,  15,  20,  30,  45, 
and  60  degree  bon*,  and  both  high  and  low 
angle  strafe.  Tactical  missions  involved 
armed  reconnaissance,  locate  and  identify 
target,  terrain  masking,  mutual  support, 


ordnance  adjustment,  moving  target,  and  FAC 
operations  in  addition  to  the  various  afore- 
mentioned delivery  modes.  During  this  inter- 
val a system  reliability  rate  of  94%  was  main- 
tained, and  after  a total  of  60  hours  system 
time  the  pilots  were  still  enthusiastic  about 
"flying"  the  simulator. 

As  a result  of  the  highly  successful  comple- 
tion of  the  engineering  modifications  and 
system  demonstration  of  the  ASUPT  phase. 
Project  2235's  major  conclusion  is  that 
computer  image  generation  is  the  most  viable 
approach  to  air-to-surface  full  mission  visual 
simulation. 
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AN  AIR  TRANSPORTABLE  PROGRAMMABLE  AIR-TO-AIR  COMBAT  SIMULATOR 
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Wright-Patterson  Air  Force  Base,  Ohio 


INTRODUCTION 

The  demand  for  increasing  use  of  flight 
simulators  within  the  military  has  led  to 
requirements  for  broader  application  and  more 
efficient  utilization. 

The  use  of  mobile  simulators  is  by  no 
means  new  to  either  the  Air  Force  or  Navy. 

Both  services  have  used  such  devices  effec- 
tively for  years;  the  Air  Force  units  being 
mounted  in  railcars  and  the  Navy  units  being 
trailerized  for  highway  transportation. 

Current  mobile  simulators  are  limited  to 
instrument  flight  simulation.  The  Air  Force 
during  the  1960s  added  visual  attachments; 
however,  this  effort  was  unsuccessful.  Recent 
advancements  in  mi ni -computers , microproces- 
sors, and  in  computer  image  generation  (CIG) 
make  mobile  simulators  in  general  and  visual 
mobile  simulators  in  particular  more  feasible 

Aerial  combat  simulation  is  potentially 
a high  payoff  application  of  mobile  simula- 
tion. This  paper  attempts  to  define  a mini- 
mal cost,  low-risk  approach  to  Air  Trans- 
portable Programmable  Air-to-Air  Combat 
Simulators  (ATPAACS). 

POTENTIAL  REQUIREMENT 

For  years,  various  organizations  have 
alluded  to  requirements  for  a mobile  air-to- 
air  combat  simulator.  The  operational  advan- 
tages of  such  a simulator  are  obvious.  This 
is  especially  true  if  one  assumes  that  such 
a simulator  has  the  flexibility  to  be  repro- 
grammed to  simulate  various  types  of  aircraft 
and  if  the  device  is  capable  of  computer 
simulation  of  an  adversary.  This  would  allow 
a single  cockpit  device  to  fly  one-on-one 
against  a preprogramed  threat.  Since  the 
proportion  of  the  flying  training  syllabus 
devoted  to  aerial  combat  may  be  rather  small, 
and  not  warrant  dedicated  aerial  combat  simu- 
lators, the  mobile  simulator  is  very  attrac- 
tive. It  is  also  highly  attractive  as  a 
tool  to  send  to  the  field  to  update  pilots  in 
the  latest  tactics  to  be  used  against  threat 
aircraft.  It  allows  a pilot  to  learn  speci- 
fic enemy  tactics  and  the  preferred  counter- 
tactics to  be  used  in  one-on-one  engagements. 
The  countertactics  may  be  chanqed  as  enemy 
tactics  change.  It  can  be  used  to  provide 
familiarization  training  with  the  perfor- 
mance characteristics  of  threat  aircraft. 


GOALS 

The  primary  goal  is  to  define  an  air-to- 
air  combat  simulator  which  is  air  transport- 
able. Ideally,  it  should  fit  into  a C-130 
aircraft  without  exceeding  its  weight  limita- 
tion for  inter-theater  shipment.  The  simula- 
tor should  have  a full  field  of  view  (FOV) 
display  (limited  only  by  aircraft  configura- 
tion) with  one  high  resolution  aircraft  image 
input  and  a horizon  reference.  As  a minimum, 
it  should  have  kinesthetic  cueing  to  include 
g-suit,  g-seat,  and  buffet  systems.  The 
aerodynamic  simulation  should  be  accurate 
and  should  be  easily  reprogrammable  to  repre- 
sent different  aircraft.  An  "iron  pilot" 
(interactive  computer  pilot)  program  to  fly 
a reprogrammable  adversary  aircraft  using 
appropriate  tactics  should  be  included.  The 
time  required  to  set  up  and  tear  down  the 
simulator  should  not  exceed  eight  hours  and 
four  hours  respectively. 

APPROACH 

Various  approaches  have  been  studied 
incluoing  both  optical  and  screen  displays, 
camera/model,  CIG,  opaque  projector  Image 
generators,  one  or  two  cockpits,  and  various 
methods  of  providing  kinesthetic  cue  simula- 
tion. Several  basic  conclusions  were 
reached.  Large  optical  displays  would  not  be 
practical.  A dome  type  screen  which  was 
inflatable  would  best  meet  the  display  and 
transportability  requirements.  Kinesthetic 
cueing  would  include  g-suit,  g-seat,  and 
buffet  systems.  Cockpit  motion  would  not  be 
practical  due  to  siting  and  size  constraints, 
setup  and  teardown  time  requirements,  the 
impractical  1 ty  of  using  motion  inside  a dome, 
and  the  problems  involved  with  mounting  a 
nonrigid  dome  on  a motion  base.  Details  of 
the  proposed  cockpit  are  provided  below. 

Cockpit.  The  cockpit  would  be  simplistic 
with  instrumentation,  systems,  and  controls 
limited  to  those  necessary  for  air-to-air 
combat.  A goal  would  be  to  provide  a conver- 
tible cockpit  configuration  with  removable 
instruments,  panels,  and  controls  to  facili- 
tate changes  to  represent  different  aircraft 
configurations.  Such  configuration  changes 
would  be  effected  at  depot  level.  Such 
changes  could  include  sidearm  controllers 
and  high  g seating  configurations. 
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Kinesthetic  Cue  Simulation.  Kinesthetic 
cue  simulation  would  include  an  active  g-suit 
system  to  provide  sustained  g-cues,  g-seat 
with  active  lap  belt  to  provide  both  onset 
and  sustained  acceleration  cues,  and  a seat 
buffet  system  to  provide  additional  aircraft 
performance  cues.  Other  methods  for  provid- 
ing kinesthetic  cueing  would  be  considered 
during  the  development  phase  of  the  program. 

Operator/ Instructor  Area.  An  operator/ 
instructor  area  would  be  provided  which  would 
stress  use  of  the  latest  Instructional  aids 
and  cathode  ray  tube  (CRT)  type  terminals  to 
provide  feedback  to  the  operator/instructor 
and  to  provide  a means  for  the  instructor  to 
brief  and  debrief  pilots. 

Image  Generation.  There  are  several 
alternatives  available  for  generating  the 
image  of  the  opposing  aircraft.  They  Include 
CIG,  camera/model  systems,  and  an  opaque  pro- 
jector system.  The  CIG  approach  is  the  most 
flexible  since  the  three-dimensional  aircraft 
image  is  digitally  stored  and  may  be  changed 
with  software  alone.  This  approach,  however, 
provides  the  least  realistic  aircraft  image. 

A camera/model  system  provides  a more  realis- 
tic image  but  is  less  flexible  and  less 
reliable.  The  opaque  projector  system  would 
involve  projecting  the  inage  from  a highly 
illuminated  gimballed  model  to  the  screen 
with  relay  optics.  This  simple,  reliable 
technique  provides  image  resolution  exceed- 
ing that  of  the  human  eye  but  is  still  less 
flexible  than  CIG,  requiring  physical  model 
changes  with  the  opaque  projector.  One  pro- 
blem that  must  be  overcone  is  that  the  low 
light  efficiency  of  the  system  may  result  in 
a dim  display. 

Display.  The  key  element  In  any  air-to- 
air  combat  simulator  is  the  visual  display. 

An  extremely  large  field  of  view  is  required. 
Two  basic  approaches  to  providing  such  a FOV 
have  been  investigated.  The  first  approach 
uses  an  inflatable  dome  similar  to  those 
used  to  protect  radar  antennas.  Such  a dome 
should  be  at  least  16  fee-t  in  diameter  with  a 
20-foot  diameter  preferred.  The  layout  in 
Figure  1 shows  an  18-foot  dome.  The  input 
to  the  display  would  be  either  a television 
or  opaque  projector.  If  the  opaque  projector 
is  determined  to  be  impractical,  hardened 
monochrome  television  projectors  are  avail- 
able which  would  be  ideal  for  mobile  use. 

The  background  display  would  be  provided 
by  a point  light  source  projector  using 
spherical  transparencies  to  provide  horizon, 
sky,  and  ground  plane  for  attitude  and  heading 
reference.  The  second  approach  Investigated 
was  a combination  helmet-mounted  sight  and  a 
helmet-mounted  display.  This  approach  would 
provide  an  unlimited  FOV  for  the  pilot  with  z 
minimum  amount  of  hardware.  It  Involves 
feeding  video  information  Including  the  threat 
aircraft  and  horizon  and  background  into  the 


helmet  display  and  controlling  position  of 
this  imagery  with  positional  information  for 
the  helmet  sight.  The  disadvantages  of  this 
approach  are  the  limited  instantaneous  FOV  of 
current  helmet  displays  and  the  Interaction 
between  information  displayed  on  the  helmet 
and  the  cockpit  structure  and  instruments. 
Current  systems  have  an  instantaneous  FOV  of 
only  25  - 40  degrees  and  only  input  Imagery 
to  one  eye.  The  limited  FOV  would  require 
more  than  normal  head  movement  by  the  viewer. 
Initial  Indications,  at  this  time,  are  that 
the  dome  display  is  more  practical  and  may 
be  lower  In  cost. 

Radar.  Radar  simulation  would  be  limited 
to  airborne  tarqets  which  would  be  generated 
digitally  using  conventional  techniques. 

Computer.  The  heart  of  the  mobile  air- 
to-air  simulator  would  be  the  computer. 
Currently  available  mini-computers  are 
extremely  compact  and  could  readily  handle 
the  simulation  required.  Initial  indications 
suggest  the  use  of  two  central  processors  and 
peripherals  including  a teletype  and  at  least 
two  magnetic  discs.  For  those  computations 
that  do  not  require  reprograming,  such  as 
integration,  linear  function  Interpolation, 
g-seat,  and  control  loading  systems,  the  use 
of  microprocessors  should  be  considered.  It 
Is  anticipated  that  with  proper  shock  mount- 
ing and  environmental  protection  it  would 
not  be  necessary  to  provide  hardened  computer 
equipment. 

Simulator  Enclosure.  The  goal  of  this 
program  would  be  to  have  the  simulator  in 
two  enclosures,  each  with  maximum  dimensions 
of  8 ft  X 8 ft  X 20  ft.  This  would  allow 
the  simulator  to  be  transported  in  either  a 
C-130  or  C-141.  Modifications  of  standard 
electronic  shelters  with  built-in  environ- 
mental control  equipment  would  probably  be 
used  to  insure  compatibility  with  Interna- 
tional Standards  Organization  (ISO)  trans- 
portability requirements.  It  may  be  desir- 
able to  include  an  auxiliary  power  unit  to 
assure  that  adequate  stable  power  is  avail- 
able for  the  simulator.  Figure  1 shows  a 
preliminary  layout  of  the  simulator  with  the 
two  enclosures  butted  together  for  operation. 
A wall  at  one  end  of  the  instructor/console 
section  is  removed  prior  to  assembly.  A 
section  of  the  enclosure  at  the  cockpit  area 
Is  removable  to  allow  Inflation  of  the 
spherical  dome.  An  air  lock  is  included  to 
allow  the  pilot  to  enter  the  cockpit  enclo- 
sure with  the  dome  Inflated. 

Training  Considerations.  To  take  maxi- 
mum advantage  of  the  mobile  simulator  con- 
cept and  to  provide  current  training,  the 
simulator  must  provide  training  against  the 
tactics  used  by  threat  aircraft  and  permit 
relatively  rapid  tactics  changes.  Threat 
aircraft  performance  is  an  Important  part  of 
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Figure  1.  Proposed  Layout  - Air  Transportable  Programnable  Air-to-Air  Combat  Simulator 


the  training  to  be  provided,  but  it  is  the 
threat  tactics,  not  aircraft  performance, 
which  are  most  subject  to  modification.  The 
interactive  target,  or  "Iron  Pilot"  should 
exhibit  threat  tactical  considerations  or 
should  permit  the  instructor  to  provide  such 
inputs  from  the  console.  The  limitations  of 
this  simulation  to  one-on-one  engagements 
limits  tactical  training  to  one-on-one  situa- 
tions but  effective  training  can  still  be 
performed.  Data  ori  threat  tactics  is  avail- 
able from  a variety  of  sources  including 
intelligence  organizations,  the  Tactical 
Fighter  Weapons  Center,  AFM  3-1  "Tactical 
Fighter  Weapons  Employment,"  and  the  F-5 
"Agressor"  Sguadrons.  The  tactics  employed 
in  a given  situation  depend  strongly  on  the 
number  of  "friendlies"  and  adversaries 
involved.  To  provide  tactical  training  with 
or  against  multiple  aircraft,  the  simulation 
of  multiple  aircraft  should  be  considered. 

LOGISTICS  CONCEPT 

Field  maintenance  would  probably  be 
limited  to  removal  and  replacement  of  cards/ 
modules.  A 30  to  90  day  spares  kit  would  be 
carried  along  with  each  mobile  trainer  to 
ensure  simulator  availability.  Repair  would 
be  accomplished  at  depot  level.  The  simu- 
lator design  will  have  to  consider  vibration, 
humidity,  and  extremes  of  heat  and  cold  to  be 


encountered  during  continuing  moves.  Design 
tradeoffs  will  be  made  between  built-in  test 
equipment  (BITE)  or  separate  supporting  test 
equipment.  The  need  for  a self-contained 
power  source  or  use  of  locally  available 
power  source  will  be  traded  off  against  the 
total  weight  and  volume  transportability  con- 
siderations. Storage  space  for  technical 
orders,  spares  kit,  test  equipment,  and  power 
source,  if  required,  would  become  part  of  the 
simulator  enclosure/shelter.  Consideration 
of  the  need  for  maintenance,  supply,  and 
instructional  areas  may  require  the  use  of 
expandable  simulator  enclosure/shelters.  Air 
transportation  to  support  movement  of  the 
simulators  will  have  to  be  provided  with  a 
priority  comnensurate  with  the  need  for  the 
training. 


SUMMARY 

A mobile  air-to-air  combat  simulator  is 
feasible  with  today's  technology.  Advance- 
ments in  computer  and  simulation  technology 
make  such  a simulator  not  only  feasible  but 
practical.  Further  study  work  should  be  con- 
ducted in  order  to  best  define  the  approach. 
The  expertise  of  several  governmental/indus- 
trial groups  seems  applicable  especially  in 
areas  such  as  inflatable  structures.  Bare 
Base  concepts,  and  iron  pilots. 
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SIGNIFICANT  FEATURES  OF  THE  UNDERGRADUATE  PILOT  TRAINING  - 
INSTRUMENT  FLIGHT  SIMULATOR  (UPT-1FS)  VISUAL/FLIGHT  SYSTEM 
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Wright-Patterson  Air  Force  Base 


INTRODUCTION 

The  USAF  Undergraduate  Pilot  Training- 
Instrument  Flight  Simulator  (UPT-IFS) 

System  design  combines  several  state-of-the- 
art  improvements  that  promise  to  provide  a 
highly  realistic  and  effective  training 
alternative  to  actual  flight  training.  A 
significant  part  of  the  UPT-IFS  system  is 
its  Visual  Subsystem  and  its  relationship 
to  the  total  simulator  complex.  Limitations 
of  previous  visual  systems  dictated  the 
need  for  establishing  minimum  image  quality 
requirements  and  associated  test  procedures 
for  the  UPT-IFS  system.  These  included  the 
utilization  of  image  detail  criteria  in 
terms  of  image  contrast  as  a function  of 
resolution  levels  of  modulation  transfer 
function  (MIF),  the  employment  of  raster 
transformation  to  achieve  the  required  low 
eye-height,  the  use  of  a high  resolution 
color  Cathode-Ray  Tube  (CRT)  in  an  infinity 
image  display,  optimized  terrain  modeling, 
and  depth-of-field  criteria. 

BASIC  UPT-IFS  PLAN 

The  basic  plan  for  the  UPT-IFS,  as 
formulated  from  a study  back  in  1968,  was 
to  acquire  a training  system  using  tech- 
nology of  proven  training  capability  and 
reliability.  Accordingly,  companion  pro- 
grams were  established  for  the  flight  sim- 
ulator system  with  Singer  Simulation  Pro- 
ducts Division  and  the  Visual  System  with 
American  Airl ines/Redifon. 

The  combined  systems  were  to  provide 
high-fidelity  training  for  all  phases  of 
instrument  flight,  including  visual  sim- 
ulation of  takeoff,  approach,  and  landing 
(or  missed  approach). 

THE  VISUAL  SYSTEM 

The  visual  system  offers  an  enhanced 
training  capability  by  providing  extra- 
instrument training  cues  for  approaches  and 
transitions  to  landings  under  variable 
weather  conditions.  It  consists  of  two 


color  television  (TV)  probe/model  image 
generators  feeding  four  cockpit  display 
crew  stations  with  infinity  image  visual 
cues  of  the  outside  world. 

These  cues  are  critical  in  that  they 
allow  practice  of  the  two-way  transition 
from  instrument  indications  to  eye-contact 
with  terrain  patterns  and  textures,  runway 
surface  markings,  lighting  configurations, 
and  signals.  Because  of  this  criticality, 
comprehensive  steps  were  taken  to  quantify 
and  qualify  the  parameters  that  provide 
these  cues.  To  provide  these  critical 
cues,  the  TV  probe/model  system  was  se- 
lected as  best  suited  for  the  requirement 
in  terms  of  desired  performance,  necessary 
flexibility,  proven  training  value  and 
reliability.  Basic  performance  advantages 
of  this  approach  were:  unprogrammed 

maneuverability,  a relatively  high  level  of 
detailed  imagery  of  day  as  well  as  night 
scenes,  and  sufficient  promise  of  providing 
the  necessary  visual  cues  in  near-the- 
ground  conditions. 

In  order  to  optimize  the  probe/model 
approach,  past  shortcomings  were  addressed 
and  special  steps  were  taken  to  effect 
improvements. 

IMAGE  QUALITY  CRITERIA 

One  of  the  principal  tasks  was  the 
adaptation  of  established  criteria  for 
assuring  and  objectively  measuring  the 
quality  of  the  displayed  image  in  terms  of 
resolution  and  contrast  transfer  (or  mod- 
ulation transfer)  characteristics.  As 
applied  to  TV  systems,  resolution  may  be 
defined  as  the  amount  of  distinguishable 
details  in  the  display,  normally  expressed 
as  the  number  of  TV  lines,  observable  in 
dimension,  equal  to  the  picture  height. 
Traditional  techniques  of  resolution  ver- 
ification have  been  mainly  eyeball,  or  sub- 
jective determinations.  In  a study  con- 
ducted by  the  Aerospace  Medical  Research 
Laboratories  (AMRL)  at  Wright-Patterson  Air 
Force  Base,  more  objective- type  measurement 
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techniques  were  developed.  These  estab- 
lished a direct  relationship  between  video 
signal  risetime  (relating  to  image-to-objert 
contrast)  and  TV  resolution  capability. 

If  a TV  system  is  required  to  repro- 
duce a gross  black-to-whi te  transition,  con- 
sisting of  resolution  bar  patterns,  the 
video  signal  waveform  would  take  virtually 
no  time  to  moveup  from  a minimum  black 
reference  level  to  the  maximum  white  re- 
sponse level.  A complete  square-wave  replica 
of  the  bar  pattern  results.  As  the  bar 
pattern  intervals  are  reduced  (to  higher 
resolution  spatial  frequencies),  and  given 
the  TV  system's  timing  constraints,  the 
move-up  time  reaches  its  practical  limit, 
at  or  before  the  given  interval  is 
traversed.  This  results  in  just  reaching 
(peakinq)  the  maximum  white  level  (100%  con- 
trast/modulation) or  partially  reaching 
that  level  (percentage  modulation).  Partial 
or  percentage  modulation,  then,  is  suitable 
as  a valuable  tool  in  measuring  resolution. 

In  adapting  this  concept  and  tech- 
nique, a typical  probe/model  candidate  system 
was  used  in  a series  of  measurements  for  per- 
formance comparisons  against  known  stan- 
dards. Measurements  of  the  system  camera's 
gray  scale,  geometric  distortion,  and  con- 
trast transfer  characteristics  were  per- 
formed through  analysis  of  the  video  output 
signals,  correlating  to  standard  and 
special  optical  test  patterns,  viewed  by 
the  camera.  The  special  patterns  consisted 
of  resolution  charts,  representative  of  four 
levels  in  spatial  frequencies  of  5,  7,  10, 
and  15  arc-minutes.  With  the  camera  set 
up  to  view  a wide  black-to-white  transition 
pattern,  a reference  amplitude  reading 
representing  the  lowest  resolution  was 
obtained,  using  a linefinding  oscilloscope 
or  waveform  monitor.  For  each  resolution 
level,  the  peak-to-peak  amplitudes  of  the 
video  signal  waveform  were  noted.  These 
amplitude  swings,  taken  as  percentages  of 
the  reference  amplitude  swing,  served  to 
reflect  the  modulation  characteristics  of 
the  system.  Measurements  were  taken  for 
each  of  the  three  color  channels,  at  nine 
different  points  within  the  field  of  view, 
and  at  simulated  ranges  of  a half-mile  and 
seven  mi les. 

After  analyzing  and  normalizing,  the 
resulting  measured  parameters,  as  shown 
in  Table  1.,  were  made  the  required  inputs 
into  a stipulated  final  display  device  (CRT 
Monitor)  of  known  characteristics  and  per- 
formance. Thus,  system  picture  quality  was 
controlled  through  specification  (and  mea- 
surement) of  both  camera  image  quality  and 
display  monitor  performance. 


DEPTH -OF-FIELD  (DOF)  RFOUIREMENTS 

In  addition  to  the  emphasis  on  the 
resol ution/MTF  parameters,  particular  atten- 
tion was  directed  to  assuring  adequate  DOF 
characteristics,  another  critical  cue 
requirement.  The  conventional  optical  pick- 
up was  favored  as  a minimum  risk  item,  even 
though  it  tended  to  have  DOF  limitations. 

To  ascertain  its  performance,  arrangements 
were  made  for  visual  systems  with  conven- 
tional pickups  to  be  demonstrated  and 
evaluated  prior  to  contract  award.  The 
results  were  considered  favorable.  Further, 
an  objective,  "Twice-the-Risetime"  method, 
(adaoted  from  the  earlier  mentioned  AWL 
study)  was  employed  to  determine  and  measure 
DOF  as  a function  of  video  signal  risetime. 
With  this  method,  an  appropriate  chart  is 
used  to  obtain  an  optimum  probe  focus  set- 
ting, corresponding  to  sharpness  in  image 
resolution.  The  video  signal  risetime  is 
taken  at  this  setting.  The  chart  is  then 
repositioned  at  points  nearer  to  and  away 
from  the  probe.  The  repositioned  points 
are  those  at  which  the  focus  is  degraded 
to  the  extent  that  the  signal  risetime  is 
not  more  than  twice  that  at  the  optimum 
focus  setting.  These  points  mark  the 
near  and  far  field  limits.  A plot  showing 
the  relationship  is  depicted  in  Figures  1 
and  2. 

USE  OF  RASTER  TRANSFORMATION 

The  optical  probe  is  the  controlling 
component  in  a TV  probe/model  visual  system. 
As  such,  the  probes  closest  approach  to  the 
model  determines  the  minimum  model  scale, 
commensurate  with  the  minimum  required  pilot 
eye-height.  State-of-the-art  probe  tech- 
nology, at  the  time,  enabled  a minimum 
probe-to-model  approach  to  be  achieved, 
that  was  sufficient  for  the  7.66  feet 
needed  for  the  T-38  application.  But  for 
the  T-37,  a lower  5-feet  eye-height  was 
required.  The  technique  of  TV  raster  trans- 
formation, or  anamorphic  compression,  was 
employed  for  reducing  the  physical  eye - 
height  to  an  apparent  5 feet.  The  trans- 
formation of  the  TV  raster  is  achieved  by 
means  of  electronic  circuitry,  driven  by 
computer  generated  height  signals,  to  change 
the  scanning  pattern  of  the  camera.  The 
picture  height,  width,  and  vertical  linear- 
ity are  changed  as  needed  to  provide  the 
necessary  aspect  ratio,  without  allowing 
unsatisfactory  side  effects,  such  as  corner 
cutting, registration  shift,  or  brightness 
changes.  A critical  aspect  of  the  trans- 
formation technique  is  to  assure  that  the 
relationship  of  apparent  distance  above  the 
terrain  to  that  of  the  actual  distance,  is 
always  greater  than  a ratio  of  0.6  to  avoid 
unrealistic  or  unsatisfactory  image  degra- 
dation when  the  video  scanning  is  modified. 


TABLE  1. 


RESOLUTION  REQUIREMENTS  FOR  VIDEO  INPUT  TO  MONITOR 


PERCENTAGE  MODULATION 
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Eor  a 36°  x 48°  FOV  on  the 

5'  corresponds  to  432  TV 

7'  corresponds  to  309  TV 

10'  corresponds  to  216  TV 

15'  corresponds  to  144  TV 


display 

lines  per  picture 
lines  per  picture 
1 ines  per  picture 
lines  per  picture 


height  resolution, 
height  resolution, 
height  resolution, 
height  resolution. 


The  rate  of  transformation  into  the  height- 
above-the-model-terrain  function  must  be  in- 
troduced gradually  and  soon  enough  to  avoid 
any  noticeable  transitioning. 

HIGH  RESOLUTION  DISPLAY  CATHODE-RAY  TUBE 

The  next  significant  feature  of  the 
UPT-IFS  visual  system  was  the  use  of  a high 
resolution,  shadow  mask  type  cathode-ray 
tube,  which  had  been  developed  on  an  earlier 
program.  The  RCA  developmental  tube, 
originally  designated  Type  C74957,  is  now 
a limited  commercial  production  item  listed 
as  RCA  1908P22.  The  center  resolution  cap- 
ability is  given  as  approximately  900  lines 
with  a white  test  pattern,  having  a bright- 
ness of  77-foot  lamberts.  Because  of  the 
prior  favorable  experience  with  this 
developmental  component  and  an  associated 
monitor  chassis,  this  display  tube  was 
specified  for  use  in  the  system.  The 
advantages  of  using  a CRT/monitor  as  image 
input  into  a mirror/beamsplitter  design, 
are  relatively  low  cost,  high  brightness, 
higher  optical  efficiency  and  less  main- 
tenance concerns,  when  compared  with  a pro- 
jector type  display  package. 

ON-AXIS  INFINITY  IMAGE  SYSTEM 

Yet  another  basic  feature  of  the  design, 
but  representing  significant  advantages 
in  realism,  is  the  60°  diagonal  field  of 
view,  on-axis  infinity  or  virtual  image  dis- 
play package.  The  infinity  image  is  pro- 
duced by  placing  the  image  input  at  the 
focal  plane  of  the  spherical  mirror.  The 
advantage,  inherent  with  a virtual  image 
display,  is  its  ability  to  present  a scene 
in  a collimated  fashion,  such  that  the  eye 
perceives  the  image  as  focused  at  infinite 
distances,  as  in  the  real  world. 

An  earlier  concern  involved  the  oper- 
ational maintenance  aspects  of  using  acrylic 
versus  glass  as  the  material  for  the  colli- 
mating mirror.  After  considering  long-range 
advantages  associated  with  glass  against  the 
more  obvious  factors  of  less  cost  and  weight 
for  acrylic,  the  former  was  selected  for  use. 
The  beamsplitter  is  also  glass. 

Indications  point  to  a compatible 
matching  of  the  curve  surfaces  of  the  high 
resolution  CRT  and  the  spherical  mirror 
design. 

MODELING  ENHANCEMENTS 

Finally,  in  keeping  with  the  basic  plan, 
existing  terrain  modeling  design  was 
optimized  to  best  suit  the  UPT-IFS  appli- 
cation. Major  efforts  centered  around  mini- 
mizing facility  size  and  power  requirements. 
Accordingly,  a common  model  scaling  (2000:1) 


was  able  to  accommodate  the  two  different 
gaming  area  (5  X 10  and  6 X 12  nautical 
miles)  requirements.  Two  different,  typical 
terrain-scapes  and  airfield  runway  configu- 
rations were  fashioned  to  provide  training 
flexibility  and  to  avoid  the  cost  and  com- 
plexity of  customized  modeling.  The  model- 
ing scheme  followed  was  to  place  emphasis  on 
realism  through  the  entire  optical  and 
video  chain  as  presented  to  the  pilot, 
rather  than  on  a realistic  looking  model 
board.  Previous  experience  had  shown  that 
model  detials,  coloration,  and  contrasts, 
for  example,  when  modeled  as  in  the  real 
world,  appear  to  be  unsatisfactory  in  the 
display  to  the  pilot. 

SPECIAL  EFFECTS 

The  integral  design  of  the  visual 
system  is  enhanced  by  a special  effects 
"fog  box,"  which  electronically  generates 
variable  visibility  restrictions,  cloud 
tops,  and  ceilings.  This  fog  box  generator 
also  provides  each  of  the  four  simulator 
cockpits  with  a capability  to  fly  into  or 
above  the  clouds,  even  when  the  probe/model 
images  are  not  available.  The  above-the- 
clouds  effects  generate  a blue  sky  above 
adjustable  shades  of  white  to  dark  gray 
cloud  tops.  The  sky/cloud  tops  “horizon" 
reference  in  the  visual  scene  is  designed 
to  be  fully  and  accurately  responsive  to 
aircraft  pitch,  roll,  and  altitude  changes. 

VISUAL  SYSTEM  PERFORMANCE 

The  UPT-IFS  visual  system  has  recently 
undergone  engineering  verification  testing. 
These  tests  were  extensive  and  included 
tests  on  the  probe/camera  combination,  the 
distribution  amplifiers  and  the  display 
system.  The  MTF  of  the  probe/camera  com- 
plied with  the  requirements  for  all  but 
a few  points.  In  many  instances,  MTF 
significantly  exceeded  the  min  m required. 
The  only  performance  area  causing  concern 
involves  limitations  in  the  on-ranwa/  DOF 
characteristics.  The  DOF  pattern  appears 
as  a band  of  reasonably  sharp  imagery  sand- 
wiched between  "softer"  image  details  in 
the  foreground  and  background.  Further 
qualification  tests  are  being  conducted  on 
the  system,  as  integrated  with  its  host 
flight  simulator,  to  assess  overall  per- 
formance adequacy. 

INTEGRATED  UPT-IFS  SYSTEM 

With  the  integration  of  the  visual 
system  with  the  "flight"  portion  of  the  UPT- 
IFS  complex,  critical  assessments  will  be 
made  as  to  the  total  systems  effectiveness 
as  originally  envisioned.  The  host  flight 
simulator  brings  as  Its  share,  several 
equally  unique,  state-of-the-art  features. 
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This  system  employs  a six-degree-of-freedom 
motion  system  for  each  cockpit,  on-board 
instructor  stations,  advanced  digital  com- 
putation systems,  automatic  demonstration 
and  aural  briefing  facilities,  a playback 
capability,  and  the  ability  to  synthesize 
emergency  conditions.  These  capabilities, 
combined  with  the  out-the-window  cues  of 
the  visua1  system,  are  expected  to  make 
a major  contribution  in  providing  high 
fidelity  synthetic  training  in  a cost- 
effective  manner  for  the  Undergraduate 
Pilot  Training  Program. 
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VERISIMILITUDE  TESTING:  A NEW  APPROACH  TO  THE  FLIGHT  TESTING  OF  AIR  FORCE  SIMULATORS 


MAJOR  JAMES  A.  RICHMOND,  USAF 
Aeronautical  Systems  Division 
Wright-Patterson  Air  Force  Base 


INTRODUCTION 

The  purpose  of  this  paper  is  to  explain 
the  mechanics  and  rationale  of  an  improved 
approach  to  simulator  flight  testing  that  the 
Air  Force  has  been  taking  with  its  most  re- 
cent simulator  procurement.  The  goal  has  been 
to  fly  and  test  the  simulator  so  as  to  iden- 
tify problems  that  it  may  have  had  in  perform- 
ance and  handling  qualities.  Problem  identi- 
fication has  been  done  in  a scientific  manner 
and  “tweaking"  has  been  avoided.  The  test 
method  that  has  been  developed  and  used,  I 
call  verisimilitude  testing. 

“What  is  verisimilitude  and  what  is  verisi- 
militude testing?"  you  ask.  Webster's  diction- 
ary! defines  verisimilitude  as  "the  appearance 
of  being  true  or  real."  It  is  just  that  - 
how  closely  the  simulator  appears  to  be  true 
or  real  - that  is  of  interest  in  the  end 
result.  Verisimilitude  testing  is  an  approach 
to  testing  the  performance  and  handling  qual- 
ities of  a simulator  that  permits  determining 
just  how  closely  the  simulator  appears  to  be 
true  or  real . 

Recognizing  that  heretofore  simulator 
testing,  within  the  Air  Force  at  least,  has 
been  a highly  subjective,  iterative  process 
commonly  referred  to  as  "tweaking,"  verisi- 
militude testing  is  a somewhat  different 
approach.  It  is  an  approach  that  seeks  to 
determine  how  closely  the  simulator  duplicates 
aircraft  performance  and  handling  qualities 
by  applying  aircraft  flight  test  techniques 
to  the  simulator. 

One  assumption  is  critical  to  this 
approach.  That  assumption  is  that  the  results 
that  are  obtained  by  using  standard  flight 
test  techniques  can  be  related  directly  to  the 
pilot's  perception  of  how  closely  a simulator 
duplicates  aircraft  performance.  Although 
there  are  undoubtedly  some  shortcomings  in 
this  assumption,  the  assumption  should  be 
basically  sound. 

REQUIREMENTS 

The  introduction  of  an  ordered  approach 
to  simulator  testing  was  considered  to  be 
necessary  because  of  the  slow,  iterative,  and 
frequently  nonrepeatable  results  of  "tweak- 
ing." The  Acceptance  Test  Procedures  written 
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under  contract  (frequently  by  the  simulator 
contractor)  were  clearly  not  the  answer.  This 
is  because  the  Acceptance  Test  Procedures  are 
written  to  tell  the  contractor  whether  or  not 
he  has  programmed  the  math  model  that  he 
thinks  he  has.  Verisimilitude  testing,  on 
the  other  hand,  is  aimed  at  determining 
whether  or  not  the  contractor  has  programmed 
a math  model  that  reflects  the  aircraft  being 
simulated.  The  big  advantages  of  verisimili- 
tude testing  over  "tweaking"  are  that  it  can 
be  used  to  compare  known  aircraft  values  to 
those  of  the  simulator,  it  can  be  accomplished 
quickly,  and  it  produces  repeatable  results. 

The  program  on  which  this  approach  to 
simulator  testing  has  been  introduced  is  the 
Undergraduate  Pilot  Training-Instrument  Flight 
Simulator  (UPT-IFS)  program.  In  this  program 
two  different  aircraft  are  being  simulated, 
the  T- 37  and  T-38,  both  of  which  are  used  to 
train  Air  Force  pilots.  The  simulators  that 
are  being  procured  have  six-degree-of-freedom 
motion  systems  and  incorporate  a terrain  model 
board  visual  system.  The  demands  of  the  Air 
Force  are  that  these  simulators  possess  a 
high  degree  of  fidelity  (or  verisimilitude!) 
with  performance  characteristics  not  "percep- 
tibly different  from  the  characteristics  of 
the  real  world  aircraft. "2  Since  the  level 
at  which  differences  become  perceptible  is 
not  specified  and  is  furthermore  not  even 
known,  the  application  of  verisimilitude 
testing  seeks  to  make  objective  comparisons 
between  the  simulator's  and  the  aircraft's 
performance  and  handling  qualities.  Areas 
where  any  differences  exist  between  the  air- 
craft and  the  simulator  can  be  more  directly 
pinpointed  and  significant  problem  areas  can 
be  made  the  subject  of  necessary  corrections. 

Insuring  that  there  are  "no  perceptible 
differences"  between  the  performance  of  the 
simulator  and  that  of  the  aircraft  is  not  an 
easy  task.  When  a simulator  is  subjected  to 
"tweaking,"  the  results  reflect  what  a small 
group  of  pilots  perceive  to  be  differences 
between  the  performance  characteristics  of  the 
simulator  and  that  of  the  aircraft.  There 
are  a number  of  problems  with  this  approach. 
Three  of  these  problems  bear  closer  scrutiny: 

a.  First,  differences  that  are  per- 
ceived may  not  exist  as  perceived.  For  ex- 
ample, a pilot  might  feel  that  control  forces 
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are  too  high  to  produce  a given  roll  rate. 
What  bothers  the  pilot  is  his  perception  of 
control  force,  but  the  problem  may  not  be  one 
of  incorrect  control  forces  at  all.  It  could 
be  that  stick  deflection  is  too  little  for  a 
given  force  or  that  aerodynamic  aileron  con- 
trol power  is  too  little.  In  other  words, 
the  pilot  can  identify  what  surfaces  in  his 
sensory  perception  as  a problem,  but  will 
likely  be  unable  to  identify  the  source  of 
such  problems. 

b.  Second,  a pilot  might  identify  a 
problem  that  is  really  not  a problem  at  all, 
or  he  may  exaggerate  the  extent  of  a per- 
ceived problem.  What  is  happening  is  that  the 
pilot's  memory  is  not  total  .and  it  is  playing 
tricks  on  him.  This  is  not  as  far  fetched  as 
one  might  imagine.  Doing  away  with  "tweaking" 
by  making  direct  objective  comparisons,  how- 
ever, easily  overcomes  this  shortcoming  of 
the  "tweaking"  approach  to  testing. 

c.  Third,  "tweaking"  does  not  produce 
highly  repeatable  results.  The  lack  of  re- 
peatability is  due  in  part  to  individual 
differences  among  pilots  and  in  part  to  the 
complex  interrelationships  of  the  various 
facets  of  simulation. 

A DIFFERENT  APPROACH 

In  order  to  avoid  the  problems  inherent 
in  "tweaking,"  the  verisimi  li  tude  approach  to 
testing  has  been  developed.  This  approach 
has  both  a quantitative  side  and  a subjec- 
tive side.  Since  the  Air  Force  was  virtually 
guaranteed  satisfaction  on  the  UPT-IFS  pro- 
gram through  the  contractual  requirement 
specifying  no  perceptible  differences  in  per- 
formance characteristics , neither  a quantita- 
tive nor  a subjective  evaluation  could  be 
ignored. 

The  objective  portion  of  verisimilitude 
testing  begins  with  a test  plan  written  to 
allow  extracting  data  from  the  simulator  that 
can  be  compared  to  available  aircraft  data. 
This  test  plan  must  be  written  to  take  advan- 
ige  of  flight  test  results  from  the  simu- 
lated aircraft.  For  the  UPT-IFS  T-37  simu- 
lator, a short  series  of  flight  tests  was 
necessary  to  fill  in  some  required  data  that 
was  not  available. 

Test  techniques  were  not  specified  in 
the  test  plan.  The  test  techniques  used, 
however,  were  consistent  with  those  used  in 
USAF  aircraft  flight  testing.3  The  test 
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pilots  detailed  to  this  project  were  thor- 
oughly familiar  with  the  use  of  these  test 
techniques  and  had  flown  test  profiles  in  an 
instrumented  T-37  aircraft  to  collect  flight 
test  data. 

Quantitative  data  collection  in  the 
simulator  was  somewhat  of  a problem.  It  was 
found  that  the  computer  output  line  printer 
was  best  used  for  some  tests  while  a strip 
recorder  was  required  for  others.  The  line 
printer  was  extremely  useful  for  testing 
level  accelerations  and  decelerations,  climbs 
and  descents,  and  longitudinal  and  lateral 
directional  static  stability.  The  strip  re- 
corder had  to  be  used  for  all  dynamic  tests, 
maneuvering  flights,  stalls,  and  spins.  Since 
the  strip  recorder  was  run  in  real-time,  it 
could  be  set  up  to  record  the  desired  para- 
meters with  optimum  scaling  and  speed.  This 
capability  greatly  facilitated  reading  data 
from  the  strip  chart. 

Once  collected,  the  data  were  reduced 
and  plotted  for  comparison  with  aircraft 
flight  test  results.  Reduction  of  simulator 
data  was  greatly  simplified  by  programing 
the  simulator  to  duplicate  aircraft  test  con- 
ditions. Reduction  to  standard  day  conditions 
was  eliminated  by  setting  the  simulated  envi- 
ronment to  standard  day  conditions.  For 
comparison  purposes,  flight  test  results  were 
scaled  and  plotted  on  graph  paper  so  as  to 
allow  the  direct  comparison  between  simulator 
and  aircraft  flight  test  results.  The  com- 
parative plots  were  the  heart  of  the  quanti- 
tative test  results  since  these  plots  were 
used  to  pinpoint  and  describe  simulator  in- 
accuracies . 

The  greatest  difficulty  in  using  this 
approach  was  interpreting  the  results.  One 
must  remember  that  the  products  of  standard 
aircraft  flight  test  techniques  are  only  re- 
flections of  the  equations  that  describe  an 
aircraft's  flight.  The  effects  of  various 
components  of  these  equations  are  frequently 
interrelated,  making  it  difficult  to  relate 
the  results  of  standard  flight  tests  to  the 
equations  of  motion  that  are  used  to  program 
a simulator.  This  difficulty  can  be  largely 
overcome  by  using  an  experienced  test  pilot 
and  flight  test  engineer  to  collect  and  in- 
terpret the  aircraft  and  simulator  flight 
test  data.  Using  conventional  flight  test 
techniques,  applied  by  an  experienced  test 
pilot  and  flight  test  engineer,  therefore, 
allows  us  to  move  from  "tweaking"  to  the  more 
orderly  and  scientific  approach  of  simulator 
verisimilitude  testing. 

Even  with  the  introduction  of  quanti- 
tative tests  for  comparison  purposes,  the  use 
of  subjective  testing  has  not  been  ignored. 
Subjective  testing  was  planned  early  in  the 
UPT-IFS  program  by  the  Air  Training  Command. 
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Valuable  assistance  in  this  area  has  been  pro- 
vided by  an  Air  Force  Institute  of  Technology 
student  who  accepted  this  project  for  his 
Master's  thesis.  In  order  to  produce  meaning- 
ful results,  it  was  necessary  to  achieve  re- 
sponse repeatability,  to  eliminate  random  re- 
sponses, and  to  insure  that  the  test  subjects 
represented  a fair  cross  section  of  Air  Train- 
ing Conmand  instructor  pilots. 

In  preparation  for  the  subjective  test- 
ing, a questionnaire  was  prepared  by  the  Air 
Training  Command  to  insure  that  each  pilot  re- 
sponded to  the  same  questions.  Two  separate 
test  missions  were  planned  for  each  pilot  par- 
ticipating in  the  subjective  evaluation.  Each 
of  these  missions  was  broken  down  into  specif- 
ic maneuvers,  and  questions  regarding  the 
perception  of  fidelity  were  written  for  each 
maneuver.  The  maneuvers  planned  included  all 
maneuvers  that  might  be  trained  in  the  simula- 
tor. Questionnaire  responses  were  structured 
so  that  each  pilot  would  be  required  to  make 
one  or  more  dichotomous  decisions  in  order  to 
rank  the  simulator's  performance. 

When  the  questionnaires  have  been  comple- 
ted and  the  results  are  compiled,  analysis  of 
the  results  will  be  accomplished  by  use  of  a 
computer  program.  The  computer  program  will 
aid  in  the  analysis  of  the  results  by  illumi- 
nating areas  where  correlations  can  be  made. 
The  correlations,  where  noted,  will  be  used 
to  determine  where  significant  problem  areas 
may  exist.  Although  few,  if  any,  unknown 
problem  areas  should  exist  following  the  quan- 
titative tesing,  the  results  of  the  subjective 
testing  should  corroborate  the  quantitative 
findings . 

The  subjective  tests  should  be  of  great- 
est utility  in  determining  where  problems 
exist  with  the  integration  of  visual  and  mo- 
tion cues  in  the  total  simulation.  The  sub- 
jective tests  may  also  provide  a valuable  in- 
put to  the  Air  Training  Command  in  structuring 
their  simulator  training  program. 

Foremost  to  note  in  the  structuring  of 
this  approach  to  simulator  testing  is  that 
"tweaking"  has  been  carefully  avoided.  This 
is  because  "tweaking"  is  an  iterative  process 
and  is  not  only  time  consuming,  but  may  also 
result  in  making  changes  that  could  affect 
the  results  of  tests  that  have  already  been 
completed.  This  does  not  mean  that  some 
obvious  problems  cannot  be  corrected  on  the 
spot,  but  problems  that  are  corrected  on  the 
spot  must  not  affect  data  that  have  already 
been  collected.  For  that  reason,  it  is  fair- 
ly critical  that  the  full  series  of  quantita- 
tive tests  be  completed  prior  to  the  correc- 
tion of  any  deficiencies.  Because  of  the 
nature  of  the  subjective  tests,  it  Is  even 
more  critical  that  the  full  series  of  subjec- 


tive tests  be  completed  prior  to  the  correc- 
tion of  any  deficiencies. 

LIMITATIONS 

Although  this  balanced  approach  to  simu- 
lator testing  is  a great  improvement  over 
"tweaking"  the  simulator  to  achieve  verisimi- 
litude or  fidelity,  there  are  also  some  dis- 
advantages. First,  there  are  no  tolerances 
to  use  as  quidelines.  Although  some  toler- 
ances should  probably  be  developed,  applying 
tolerances  indiscriminately  would  be  almost 
meaningless.  Therefore,  the  opinion  of  a 
trained  test  pilot  must  be  used  in  lieu  of 
arbitrary  tolerances  when  evaluating  the  per- 
formance and  handling  qualities  of  a simula- 
tor. 

Second,  this  approach  to  testing  a simu- 
lator cannot  accurately  probe  the  pilot's 
perception  of  the  programmed  math  model. 

What  is  meant  is  that  the  perceived  stability 
and  control  derivatives  may  be  different  than 
those  that  are  programmed.  Because  of  the 
time  delays  that  exist  from  pilot  input  to 
total  system  response  and  because  of  such 
factors  as  motion  system  washout,  visual  sys- 
tem scaling,  and  control  loading  system  in- 
accuracies, it  is  likely  that  there  is  a con- 
siderable difference  between  the  model  that 
the  pilot  perceives  in  a simulator  and  what 
he  would  perceive  in  the  aircraft  with 
virtually  the  same  equations  of  motion.  For 
this  reason,  an  accurate  math  model  may  not 
always  produce  an  accurate  simulation.  A 
meaningful  test  that  would  yield  the  compo- 
nents of  the  perceived  math  model  is  not 
available.  Therefore,  an  impasse  could  be 
reached  if  the  subjective  feel  is  that  the 
fidelity  of  the  simulator  is  not  satisfactory 
but  the  quantitative  tests  yield  no  signifi- 
cant problems. 

The  above  two  limitations  appear  to  be 
the  most  significant,  but  by  no  means  the 
only  limitations  of  verisimilitude  testing. 
For  example,  tests  requiring  outside  refer- 
ences for  optimum  performance  are  constrained 
by  the  limitations  of  the  visual  system. 

These  limitations  on  verisimilitude  testing, 
however,  should  not  be  overemphasized. 

TEST  PROGRESS 

The  first  round  of  quantitative  tests 
have  already  been  completed  on  the  UPT-IFS 
T-37  simulator  and  the  data  have  been  analyz- 
ed. The  analyses  detected  evidence  of  comnon 
problems  turning  up  in  more  than  one  test. 
This  recurrence  of  certain  common  problems 
helped  to  pinpoint  specific  problems.  Once 
the  deficiencies  have  been  noted  and  correct- 
ed, round  two  of  the  quantitative  testing  can 
take  place.  Following  the  correction  of  the 
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deficiencies  noted  in  round  two  of  the  quan- 
titative testing,  the  subjective  tests  can 
begin.  Should  any  deficiencies  remain  at  the 
commencement  of  the  subjective  testing,  the 
results  of  the  subjective  tests  should  either 
confirm  or  deny  the  severity  of  any  such  pro- 
blems. This  should  provide  a form  of  check 
and  balance  concerning  the  necessity  of  cer- 
tain corrective  actions. 

Where  will  verisimilitude  testing  go 
after  it  has  been  conducted  on  the  T-37  and 


T-38  simulators  being  procured  under  the 
UPT-IFS  program?  Thus  far  this  approach  to 
simulator  flight  testing  has  shown  great 
promise.  Given  its  continued  success  as  an 
efficient  means  to  test  the  fidelity  of 
simulator  performance  and  handling  qualities, 
it  will  likely  be  used  on  other  simulator 
procurements  now  in  progress.  Verisimili- 
tude testing  is  a very  flexible  approach  to 
simulator  flight  testing  and  can  be  easily 
tailored  to  meet  the  demands  of  the  situa- 
tion. 
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BACKGROUND 

Since  the  early  days  of  flying,  flight 
simulators  have  been  in  use  and  their  value 
has  been  amply  demonstrated.  Flight  simu- 
lators have  evolved  from  the  Link  Trainer, 
which  was  widely  used  for  pilot  training  in 
the  Second  World  War,  into  precisely  en- 
gineered devices  capable  of  accurately  com- 
puting the  aerodynamic  responses  of  an  air- 
plane to  control  inputs,  and  of  reproducing 
realistic  cockpit  instrument  indications 
for  all  flight  situations.  It  was  realized, 
as  Adams  (1957)  points  out,  that  flight 
simulators  have  many  advantages  over  the 
operational  situation.  First,  the  simula- 
tor provides  its  users  with  greater  control 
over  ambient  conditions.  Whereas  the  "real" 
world  is  subject  to  unpredictable  varia- 
tions, a simulator  can  provide  planned 
variation  of  various  elements  of  the  real 
situation  with  unessential  elements  in  the 
real  situation  omitted.  Second,  the  simu- 
lator can  represent  dangerous  elements  in 
flight  more  safely.  Emergency  procedures 
that  would  be  too  dangerous  to  teach  in  the 
air  may  be  taught  safely  in  ground-based 
simulators.  Third,  a major  advantage 
offered  by  simulators  is  their  low  operating 
costs  in  comparison  with  the  costs  of 
operating  aircraft.  For  tnese  reasons, 
simulators  continue  to  play  an  important 
role  in  pilot  training  during  initial  ac- 
quisition, transition  training,  and  for 
maintenance  of  established  flying  skills. 

The  importance  attached  to  simulators  in 
meeting  training  goals  is,  of  course,  pred- 
icated on  the  assumption  that  training 
given  in  the  simulator  will  transfer  to  the 
aircraft. 

Although  there  has  never  been  a serious 
challenge  concerning  the  value  of  simulators 
to  pilot  training,  the  utility  of  simulator 
training  for  maintaining  the  proficiency  of 
experienced  pilots  has  not  been  adequately 
established.  The  various  studies  that  have 
assessed  the  training  effectiveness  of  a 
specific  flight  simulator  for  maintaining 
pilot  proficiency  among  experienced  pilots 
have  sometimes  produced  contradictory  re- 
sults (Caro,  1971;  Crook,  1965;  American 
Airlines,  Inc.,  1969;  TransWorld  Airlines, 
1969).  For  example,  whereas  Caro  reports 
positive  transfer  from  the  simulator  to 


the  aircraft  among  experienced  pilots,  Crook 
reports  less  conclusive  findings.  Three 
somewhat  different  explanations  for  these 
discrepant  findings  have  been  suggested. 
First,  the  variance  in  flying  skill  among 
experienced  pilots  is  often  greater  than 
among  student  pilots  who  bring  little,  if 
any,  initial  skill  to  the  task  (McGrath 
and  Harris,  1971).  In  performance  training 
it  is  well  known  that  the  beneficial 
effects  of  training  varies  as  a function 
of  experience  or  skill  level  (Briggs, 

1957;  Roscoe,  1971);  that  is,  the  same 
training  simulator  may  exhibit  different 
effectiveness  functions  for  different 
levels  of  pilot  experience  or  skill.  To 
the  extent  that  experienced  pilots  are  not 
carefully  matched  in  terms  of  flying 
skill,  the  significance  of  assessing  what 
can  be  learned  in  the  simulator  is  dimin- 
ished. Second,  most  performance  measures 
used  were  judgmental  in  nature  and,  thus, 
highly  subjective  evaluation  instruments. 
Third,  instructional  techniques  may  vary 
widely  depending  upon  the  ability  and 
attitudes  of  the  instructor  pilots.  In 
considering  the  possibilities  of  such 
instructional  differences,  it  is  essentia] 
to  determine  the  utility  of  simulator 
training  independent  of  the  instructor 
variable. 

Currently,  there  is  a need  to  deter- 
mine the  training  value  of  Device  2B24,  a 
high-fidelity  simulator,  for  maintaining 
instrument  flight  (IFR)  proficiency  among 
experienced  Army  helicopter  pilots  in  the 
'H-1H  aircraft.  This  paper  reports  on  the 
transfer  effectiveness  of  Device  2B24  for 
use  in  maintaining  IFR  skills  among  ex- 
perienced pilots  as  compared  to  inflight 
training.  As  will  be  seen  below,  this 
study  extends  previous  studies  by  more 
adequate  control  of  pilot  and  instructor 
variability  and  through  more  objective 
measures  of  pilot  performance  obtained 
both  before  and  after  training  in  Device 
2B24. 

METHOD 

Pi  lots . The  pilots  participating  in 
this  study were  36  fully  qualified,  combat 
ready  Army  aviators.  The  sample  was  com- 
prised of  both  warrant  and  commissioned 
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officers,  all  of  whom  had  between  400-600 
hours  of  rotary  wing  flight  experience. 

Two  groups  of  aviators  of  18  pilots  each 
were  formed  on  the  basis  of  a composite 
score  on  three  IFR  performance  measures. 

A written  test  on  instrument  procedures  (I), 
a standardized  IFR  check  ride  in  the  UH-1H 
helicopter  (H),  and  a standardized  IFR  check 
ride  in  Device  2B24  (S)  were  the  three 
measures  making  up  the  composite  score 
(P).  The  a priori  scoring  formula  reads: 

P = .51  + S + H.  The  three  performance 
measures  were  administered  to  each  of  51 
experienced  pilots  assigned  to  the  101st 
Aviation  Group,  Ft.  Campbell,  Kentucky. 

The  best  18  and  the  poorest  18  scores  were 
used  for  group  matching  and  for  control 
of  differences  in  initial  ability.  The 
two  groups  are  referred  to  as  "high"  and 
"low,"  respectively. 

Following  the  selection  of  each  group, 
pilots  from  each  group  were  quasi -randomly 
assigned  to  one  of  three  IFR  training  modes 
under  the  following  conditions: 

a.  Simulator  training  with  Device 

2B24. 

b.  Inflight  training  in  the  UH-1H. 

c.  A combination  of  simulator  and 
inflight  training  divided  equally. 

A total  of  12  pilots,  6 from  each  group, 
were  assigned  to  each  training  mode.  This 
procedure  provided  an  opportunity  not  only 
to  evaluate  training  modes  but  to  evaluate 
them  with  respect  to  initial  skill  level 
among  pilots.  An  analysis  of  variance 
failed  to  show  reliable  differences  between 
pilots  assigned  to  each  training  mode 
( F (2 ,30)  ■ .31),  but  revealed  a reliable 
difference  between  the  pilot  groups  assigned 
within  each  mode  { F ( 1 , 30)  - 17.91,  p < .001) 
as  constituted.  Thus,  pilots  appeared  to  be 
effectively  matched  among  training  modes 
while  differentiated  in  terms  of  initial  IFR 
skills. 

Procedure.  The  pilots  in  each  of  the 
three  training  modes  were  given  a total  of 
12  hours  of  IFR  training  during  9 
months  of  testing  in  each  of  the  three 
training  modes--that  is,  in  the  SFTS,  in- 
flight, or  combined  SFTS  and  inflight 
training.*  The  12  hours  of  training  were 
divided  into  4 hourly  increments  taken 
uarterly.  At  the  end  of  each  quarterly 
3 month)  period,  pilots  were  required  to 
take  a standardized  IFR  check  ride  in  the 
UH-1H,  identical  in  all  its  essentials 


with  the  IFR  check  nde  given  before  train- 
ing. 

Prior  to  testing,  an  objective  in- 
ventory for  pilot  evaluation  was  developed 
at  the  Ft.  Campbell  facility.  The  de- 
cisions to  include  items  were  based  upon 
desired  terminal  flight  behavior  for  ex- 
perienced Army  pilots.  Evaluated  tasks 
were  arranged  into  a standard  flight  se- 
quence, including  both  objective  and  sub- 
jective items.  The  inventory  evaluated  per- 
formance on  each  of  five  quantitative  var- 
iables with  one  to  four  items  for  each 
variable  scored  on  an  appropriate  scale. 
Equal  weights  were  given  to  each  of  the 
objective  variables  measured;  namely:  (1) 

flight  planning,  (2)  departure  and  enroute 
procedures  (e.g.  , navigation  and  tracking, 
radio  reports),  (3)  holding  and  arrival 
procedures  (e.g.  , missed  approach,  time, 
track,  airspeed,  etc.),  (4)  ATC  procedures, 
and  (5)  aircraft  control  (e.g.,  airspeed, 
altitude,  heading,  etc.).  All  scored 
items  were  completed  during  flight.  A 
score  of  100  represented  "errorless" 
performance. 

To  minimize  the  effects  of  instructor 
differences,  six  instructor  pilots  (IP) 
were  given  familiarization  training  on  the 
use  of  the  flight  inventory  before  each 
check  ride.  The  order  in  which  IP's  were 
assigned  to  provide  check  rides  was  varied 
systematically.  In  addition,  knowledge 
of  an  individual  pilot's  ranking  and  group 
assignment  was  withheld  from  the  IP's  to 
prevent  such  information  from  influencing 
their  judgment.  Nevertheless,  because 
of  the  relatively  small  pilot  sample,  some 
IP’s  were  aware  of  a particular  pilot's 
group  assignment. 

Instrument  flight  training  began  soon 
after  the  two  groups  of  pilots  were  formed. 
Training  sessions  were  1 hour  in  length 
and  four  sessions  completed  a quarterly 
training  period.  At  the  end  of  each  quarter 
of  training,  an  instrument  check  ride  In 
the  UH-1H  was  given  to  each  pilot  using 
the  flight  inventory  for  scoring  pilot 
performance.  The  present  paper  reports 
on  some  initial  findings  of  the  trans- 
fer effectiveness  of  Device  2B24  among  ex- 
perienced pilots. 

RESULTS 

Because  of  conditions  not  related  to 
the  experimental  variables,  two  of  our 
pilots  assigned  to  the  "low"  group,  one 
in  the  SFTS  training  mode  and  the  other 


*The  experiment  was  originally  designed  to  provide  16  hours  of  training  divided  into  four  quar- 
terly increments.  We  were  unable  to  complete  the  last  quarterly  test  period  due  to  the  tem- 
porary transfer  of  the  pilots  to  a new  duty  station. 
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in  the  combined  training  mode  were  unable 
to  complete  the  experiment  during  the 
third  quarter  testing  period.  The  third 
quarter  results  and  the  statistical  anal- 
yses are  based  then  on  an  unequal  number  of 
pilots  among  blocks. 

The  main  results  deal  with  the  dif- 
ferences in  check- ride  performance  fol- 
lowing training  in  the  three  training  modes 
and  the  percent  change  in  performance 


representing  the  differences  between  check- 
ride  performance  obtained  before  and  after 
training.  The  curves  in  Figure  1 compare 
mean  inflight  performance  for  the  three 
training  modes  over  the  three  test  periods 
with  baseline  performance  plotted  separately. 
The  curves  show  that  over  test  periods  the 
changes  due  to  training  were  not  the  same 
for  each  group,  although  the  relative  level 
of  performance  for  the  different  training 
modes  persisted  throughout  training.  That 


INFLIGHT 
TEST  SCORES 


INFLIGHT 
TEST  SCORES 


Figure  1.  Learning  Curves  of  Check-ride  Performance  for  the 
Three  Training  Modes  and  Two  Pilot  Groups. 
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is,  while  the  three  curves  for  the  "low" 
group  showed  marked  learning  effects  over 
test  periods,  the"high"  group  gave  very 
little  evidence  of  learning.  Both  groups 
however,  showed  that  check-ride  perfor- 
mance was,  in  general,  best  for  Device 
2B24  training  and  poorest  for  inflight 
training.  Analysis  of  variance  for  re- 
peated measures  indicate  that  variation  in 
performance  due  to  test  periods  was 
significant,  F(2,46)  = 17.9,  p < .01,  for 
the  "low"  group,  although  not  a significant 
source  of  variance  for  the  “high"  group. 

It  is  also  important  to  note  that  the 
analysis  revealed  that  the  differences 
between  pilots  assigned  to  groups  were  no 
longer  statistically  significant  following 
training.  Further,  although  Device  2B24 
training  appeared  to  result  in  better 
performance  throughout  the  training  period, 

F tests  of  the  training  mode  means  for 
each  group  were  not  significant.  However, 
Duncan  multiple  range  tests  (Duncan,  1955) 
were  carried  out  on  the  training  mode 
means  of  the  last  training  period  to 
determine  which,  if  any,  of  the  differences 
between  the  means  are  significant  and 
which  are  not.  Results  of  these  tests  are 
sunmarized  in  Table  1.  Common  lines 
underlying  two  or  more  means  indicate  that 
these  means  are  not  significantly  dif- 
ferent from  one  another.  The  results  on 
means  for  the  third  training  period  indicate 
that  for  the  low  group  the  mean  differences 
between  inflight  training  and  Device  2B24 


training  were  significantly  different. 

All  other  comparisons  were  not  significant. 
Thus,  the  value  of  Device  2B24  is  particu- 
larly apparent  for  the  low  group. 

Table  1 


RESULTS  OF  RANGE  TEST  OF  THE  DIFFERENCES 
BETWEEN  PERFORMANCE  SCORE  MEANS  FOR 
TRAINING  MODE  AND  PILOT  GROUP 


Note:  Significant  at  the  .05  level.  Com- 

mon lines  underlying  two  or  more  means  in- 
dicate that  these  means  are  not  signifi- 
cantly different  from  one  another. 

Percent  change  scores  for  the  three 
training  modes  are  given  by  the  curves  of 
Figure  2.  These  curves  plot  the  difference, 
in  percent,  between  check-ride  performance 
before  and  after  training  in  each  mode  and 
each  test  period.  The  curves  show  again 
that  the  level  of  change  in  check-ride  per- 
formance were  quite  distinct,  with  the 
SFTS  mode  producing  the  highest  degree  of 
positive  transfer  for  both  groups.  The 


% CHANGE 


TEST  PERI00 

Figure  2.  Change  Scores  in  Check-ride  Performance  for  the  Two 
Pilot  Groups  as  a Function  of  Training  Mode  and 
Test  Periods. 


plot  of  these  test  period  scores  show  that 
most  of  the  performance  changes  that  oc- 
curred took  place  by  the  second  test  period. 
It  may  be  noted,  for  example,  that  all  of 
the  transfer  that  occurred  for  Device  2B24 
training  occurred  by  the  second  test  period; 
that  is,  after  8 hours  of  training  in  Device 
2B24.  Thus,  the  loss  of  the  fourth  test 
period  did  not  appear  to  be  crucial  to  the 
present  interpretation  of  our  results. 

A three-way  analysis  of  variance,  en- 
compassing groups,  training  modes  and  test 
periods,  was  carried  out  on  the  change  score 
data.  Results  of  the  analysis  related  to 
the  principal  variables,  and  their  first 
order  interactions  indicate  that  only  the 
variations  due  to  groups  and  test  periods 
were  statistically  significant  at  the  .025 
and  .01  levels,  respectively. 

Duncan  multiple-range  tests  were  again 
carried  out  on  the  training  mode  change 
scores  for  the  third  measurement  period. 

The  results  are  shown  in  Table  2. 

Table  2 


RESULTS  OF  RANGE  TESTS  OF  THE  DIFFERENCES 
BETWEEN  PERCENT  CHANGE  SCORE  MEANS  FOR 
TRAINING  MODE  AND  PILOT  GROUP 


Group 

SFTS 

Combined 

Inflight 

High 

5 

-4 

-7 

Low 

28 

27 

14 

maintained  or  even  enhanced  subsequent  per- 
formance in  the  UH-1H  among  experienced 
pilots.  In  this  research  the  transfer  ef- 
fectiveness of  the  simulator  is  measured  by 
determining  the  extent  to  which  performance 
of  Device  2B24  trained  groups  are  equal  to 
or  exceed  the  performance  of  the  inflight 
trained  group  on  the  criterion  check  ride. 

The  results  of  this  study  indicate 
substantial  positive  transfer  of  training 
from  Device  2B24  to  the  UH-1H  helicopter 
as  evidenced  by  the  fact  that  simulator 
training  produced  as  good  or  better  over- 
all performance  as  inflight  training. 

Thus  , Device  2B24  training  appeared  to  be 
as  much  a factor  in  maintaining  IFR  skills 
among  experienced  Army  pilots  as  inflight 
training  for  the  periods  of  time  tested. 
These  findings  were  characteristic  of  both 
groups  of  pilots.  Further,  the  results 
showed  that  the  beneficial  effects  of 
training  were  more  often  observed  among 
pilots  who  demonstrated  the  least  IFR 
skill  level  as  measured  by  our  criterion 
check  ride.  The  consequences  of  our 
training  procedures  were  that  the  initial 
differences  in  IFR  performance  between  our 
two  groups  of  pilots  were  no  longer 
significant. 

The  fact  that  IFR  training  in  Device 
2B24  transfers  to  the  UH-1H  is  not  surpris- 
ing since  it  is  a high-fidelity  simulator 
of  the  UH-1H  helicopter.  These  results 
are  also  consistent  with  the  record  of 
commercial  aviation  which  has  shown  that 
simulators  can  provide  an  effective  means 
of  providing  training  for  highly  exper- 
ienced pilots. 


Note:  Significant  at  the  .05  level.  Com- 

mon lines  underlying  two  or  more  means  in- 
dicate that  these  means  are  not  signifi- 
cantly different  from  one  another. 

negative  values  in  Table  2 indicate  a per- 
centage decrement  in  performance  associated 
with  training.  This  decrement,  we  believe, 
is  in  accord  with  normal  variation  in 
performance  that  may.be  expected  with  a 
pilot  group  already  at  a high  level  of  IFR 
proficiency  prior  to  training.  The  results 
on  change  scores  indicate  that  Device  2B24 
training  and  inflight  training  show  the 
greatest  number  of  significant  differences 
consistent  with  the  evidence  of  beneficial 
effects  for  SFTS  training  when  compared  to 
inflight  training. 


It  should  be  emphasized,  however,  that 
the  effectiveness  of  any  training  device 
depends  upon  how  it  is  used;  it  is  in- 
fluenced by  all  the  well-known  variables 
concerning  conditions  favorable  and  un- 
favorable to  learning.  Prophet  (1966) 
has  pointed  out  that  a simulator  is  a 
training  tool  and,  as  such,  is  going  to 
produce  results  no  better  than  the  quality 
of  the  training  program  of  which  it  is  a 
part.  The  evidence  of  transfer  in  the 
present  study  may  well  be  attributed  to 
the  training  program  rather  than  simulator 
design  alone.  It  is  possible,  for  example, 
that  if  inflight  training  was  conducted  dif 
ferently--that  is,  with  a different  train- 
ing program--some  evidence  of  additional 
benefits  might  be  found. 


DISCUSSION 

The  objective  of  this  research  was  to 
determine  the  extent  to  which  IFR  skills 
acquired  during  simulator  training  in 
Device  2B24,  or  inflight,  or  In  combination. 


The  present  findings  complement  past 
results  which  demonstrate  the  operational 
utility  of  simulator  training  (Caro,  1972; 
Woodruff  and  Smith,  1974).  In  particular, 
this  preliminary  analysis  of  the  data  in- 
dicate that  much  of  the  Instrument  training 
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now  conducted  in  rotary  wing  aircraft 
can  be  conducted  more  efficiently  on  the 
ground.  Although  with  Device  2824  pilots 
may  require  approximately  the  same  number 
of  hours  training  as  in  the  aircraft,  the 
total  time  required  is  actually  reduced  by 
reason  of  the  greater  availability  of  the 
simulator  and  the  elimination  of  much  of 
the  aircraft  preparation  time.  The  safety 
associated  with  elimination  of  much  of  the 
inflight  training  time,  the  release  of 
aircraft  for  operational  flights,  and 
decreased  cost  of  operating  the  simulator 
rather  than  an  aircraft  for  training,  all 
add  up  to  a highly  positive  evaluation  of 
Device  2824  for  the  maintenance  of  1FR 
pilot  proficiency.  A reasonable  conclu- 
sion from  this  study  is  that  some  evidence 
was  found  to  support  the  assumption  that 
simulator  training  administered  in  Device 
2B24  can  maintain,  and  perhaps  improve, 
subsequent  pilot  performance  in  a tactical 
instrument  situation.  Whether  or  not  the 
attitudes  of  the  pilots  who  served  as  our 
subjects  are  in  accord  with  this  inter- 
pretation, is  presently  under  review. 
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COMBAT  READINESS  THROUGH  ENGAGEMENT  SIMULATION 


LTC  GEORGE  J.  STAPLETON,  US  ARMY 
Program  Manager,  Engagement  Simulation 
US  Army  Training  Support  Center 
Fort  Eustis,  Virginia 


For  nearly  three  years  the  US  Army  has 
been  experimenting  with,  refining,  and  im- 
plementing a series  of  training  techniques 
which  employ  engagement  simulation  mecha- 
nisms, along  with  proven  instructional 
models,  to  improve  unit  tactical  training. 
Some  see  the  advent  of  engagement  simulation 
as  merely  another  step  along  the  Army's  path 
toward  increased  realism  in  training.  Others 
see  engagement  simulation  as  a near  perfect 
simulation  of  combat,  while  still  others 
believe  engagement  simulation  is  nothing  new, 
and  that  in  one  form  or  another,  "we  have 
been  doing  this  kind  of  thing  for  years." 
Probably  none  of  the  foregoing  opinions  on 
engagement  simulation  are  entirely  right, 
nor  are  any  totally  incorrect.  More 
properly,  the  techniques  at  issue  certainly 
simulate  the  violent  interactions  of  weapons 
in  combat,  although  imperfectly,  and  in  so 
doing  add  much  realism  to  our  tactical 
training.  While  we  can  point  to  excel- 
lent training  with  conventional  techniques, 
some  of  the  aspects  of  engagement  simulation 
are  truly  new,  or  are  at  least  substantial 
revisions  of  earlier  ways  of  doing  things. 

The  main  thrust  of  engagement  simulation 
has  been  directed  toward  small  unit  tactical 
training.  Tactical  training  is  a complex 
undertaking  which  we  have  normally  handled 
by  attempting  to  break  it  down  into  its 
lowest  common  denominator  or  component  tasks: 
firing  a weapon,  operating  a radio,  writing 
an  order,  or  adjusting  artillery  fire.  We 
train  on  these  component  tasks,  and  presume 
that  in  combat  we  can  sum  them  into  an  effec- 
tive whole.  But  until  very  recently  we  have 
had  no  means,  short  of  combat,  of  actually 
testing  our  summations.  Engagement  simula- 
tion provides  us  one  means  of  doing  just 
that. 

In  an  academic  manner,  engagement  simu- 
lation can  be  defined  as  the  employment  Of 
systems  or  devices  to  simulate  with  a high 
degree  of  fidelity  the  casualty  producing 
effects  of  weapons  found  on  the  modern 
battlefield,  during  two-sided,  free-play. 


tactical  exercises.  But  what  do  we  really 
mean  by  engagement  simulation?  In  simple 
terms,  we  seek  a believable  way  to  say  to 
the  soldier  in  tactical  training  exercises, 
"You  exposed  yourself  on  this  training 
battlefield  in  such  a manner  that  an  opposing 
soldier  was  able  to  draw  a bead  on  you  with 
his  rifle,  or  put  artillery  fire  on  you,  or 
with  some  other  weapon,  has  caused  you  to 
become  a casualty."  For  engagement  simula- 
tion to  be  effective,  the  soldier  must  accept 
two  things:  first,  he  did  indeed  expose  him- 
self and  get  "hit,"  and  second,  this  training 
occurrence  has  a one-for-one  correspondence 
to  combat.  In  his  own  mind  the  soldier  must 
forge  the  link  between  a "casualty"  in 
training  and  in  combat.  When  we  thus  con- 
front him  in  a credible  manner,  he  responds 
as  he  seldom  did  to  other  training.  Our 
initial  field  evaluations  of  engagement  simu- 
lation clearly  establish  that  without  further 
prompting,  the  average  soldier  will  recognize 
his  error,  and  take  action  to  preclude  a 
recurrence  of  the  mistake  that  led  to  his 
being  "hit."  While  he  will  surely  make 
other  mistakes  in  other  exercises,  he  rapidly 
learns  through  additional  repetitions  the 
fundamental  lessons  of  combat,  and  modifies 
his  behavior  in  a manner  that  influences  and 
increases  his  chances  of  survival.  When  we 
have  soldiers  with  the  skills  to  survive  in 
combat  who  use  concealment,  exploit  all 
available  cover,  and  employ  suppressive  fire 
to  facilitate  movement,  we  can  turn  our 
attention  to  training  small  tactical  units 
in  teamwork. 

The  objective  of  small  unit  tactical 
training  is  to  provide  combat  units  with  the 
skills  required  to  fight  and  survive  on  the 
modern  battlefield.  It  differs  from  other 
types  of  training  in  that  it  must  realisti- 
cally simulate  the  combat  environment.  This 
environment  involves  the  aforementioned 
violent  interaction  of  two  opposing  forces 
who  are  out  to  destroy  one  another.  To 
prepare  our  combat  units  to  operate  effec- 
tively in  this  environment,  it  is  necessary 
to  train  them  to  operate  against  a realistic 


opposing  force.  It  is  not  sufficient  to 
merely  train  our  units  to  fire  at  the  enemy; 
we  must  train  them  to  fire  and  move  against 
an  enemy  which  is  firing  back,  and  doing 
everything  in  its  power  to  neutralize  or 
avoid  our  fire. 

Perhaps  the  contribution  that  engagement 
simulation  makes  to  tactical  training  is  most 
easily  understood  by  comparing  it  with  tradi- 
tional forms  of  combat  training.  Up  until 
recently  all  of  our  tactical  field  exercises 
took  one  of  two  forms  — live  ammunition 
against  cardboard  targets,  or  blank  arnnuni- 
tion  fired  toward  a live  opponent. 

The  effectiveness  of  live  fire  exercises 
against  cardboard  targets  is  somewhat  decep- 
tive. They  often  sound,  smell,  and  look  so 
much  like  combat  that  commanders  are  lulled 
into  assuming  they  realistically  simulate 
combat.  Careful  analysis,  however,  shows 
that  they  bear  only  a superficial  resem- 
blance to  battle.  Cardboard  targets  can  not 
shoot  back  and  hence  scarcely  represent  the 
skilled  and  determined  enemy  we  will  face  in 
combat;  they  don't  try  to  nide  themselves 
from  us,  and  they  don't  try  to  suppress  our 
fire.  As  a result,  soldiers  well  trained 
with  live  fire  exercises  often  encounter  a 
tremendous  shock  when  they  first  encounter 
real  opposition,  not  because  of  the  noise  of 
combat,  but  because  they  are  opposed  by  a 
determined  enemy  which  they  almost  never  see, 
which  is  trying  to  kill  them,  and  which  seems 
almost  impervious  to  their  fire. 

Blank  fire  exercises  enable  us  to  deal 
more  effectively  with  the  enemy  because  they 
permit  two  opposing  forces  to  maneuver 
against  one  another,  signaling  an  engagement 
by  shooting  blanks.  However,  while  the  blank 
rounds  realistically  simulate  the  act  of 
firing,  they  provide  no  indication  whatsoever 
of  the  effectiveness  of  the  fire.  This 
determination  is  left  entirely  to  the  subjec- 
tive judgement  of  the  NCO  or  officer  who  is 
controlling  the  exercise.  Since  he  will 
normally  be  required  to  deal  with  many  sol- 
diers, and  a relatively  large  number  of 
diverse  weapons  in  a fast  moving,  complex 
situation,  and  since  he  can  often  see  only  a 
portion  of  the  entire  action  at  any  one  time, 
his  judgements  are  frequently  quite  general 
in  nature--"this  side  won  because  it  was  four 
times  larger"  or  "that  side  won  because  it 
had  better  cover  and  concealment."  While 
information  of  this  type  is  of  some  value  in 
training  combat  units,  it  falls  far  short  of 
that  required  to  develop  truly  skilled  and 
battlewise  soldiers. 

Engagement  simulation  training  systems 
overcome  the  shortcomings  of  both  live  and 
blank  fire  training  methods.  Engagement 


simulation  consists  of  three  steps,  each  of 
which  encompasses  a singularly  important,  and 
essential,  training  function. 

Step  1 involves  realistically  exposing 
the  soldier  to  the  lethality  of  weapons.  We 
equip  each  soldier's  weapon,  each  armored 
vehicle,  and  each  weapon  crew  with  devices 
which  realistically  simulate  its  casualty 
producing  capability.  Once  so  equipped,  two 
opposing  forces  conduct  a free-play  tactical 
exercise  under  no  more,  or  no  less,  con- 
straints than  exist  on  an  actual  battlefield. 
The  employment  of  the  simulation  devices 
enables  the  battle  to  unfold  exactly  as  it 
would  in  combat.  When  a soldier  fires  and 
hits  another  soldier,  the  soldier  who  is  hit 
receives  an  immediate  indication  he  has  made 
a mistake,  and  is  "out  of  action."  Likewise, 
if  the  soldier  who  is  firing  does  not  take 
adequate  precautions,  he  can  be  hit  by  return 
fire.  In  this  manner,  each  soldier  taking 
part  in  the  engagement  simulation  exercise 
receives  immediate  feedback  concerning  his 
actions.  If  he  uses  proper  techniques  and 
tactics  in  each  situation  he  encounters,  he 
continues  to  contribute  to  his  team;  if  he 
does  not,  he  is  eliminated.  Immediate  feed- 
back is  an  essential  element  of  all  effective 
training  systems;  but  until  the  development 
of  engagement  simulation  training  systems, 
we  had  no  reliable  way  of  obtaining  it  during 
tactical  training. 

Step  2 involves  confronting  each  soldier 
with  a critique  of  his  actions,  and  begins 
at  the  completion  of  the  two-sided,  free- 
play  tactical  exercise.  We  bring  both  forces 
together  and  conduct  a detailed  afteraction 
review.  During  this  review,  each  soldier  who 
has  hit  an  opponent  explains  in  detail  how 
he  was  able  to  detect  and  engage  him.  This 
is  followed  by  a mutual  discussion  of  what 
the  soldier  who  was  hit  could  have  done  to 
prevent  this  happening.  In  this  manner  the 
techniques  and  tactics  which  enable  soldiers 
to  succeed  are  identified  and  discussed. 
Unlike  traditional  critiques  which  normally 
only  involve  the  officer  or  NCO  in  charge 
lecturing  the  soldiers  on  mistakes  he 
observed,  these  afteraction  reviews  get  all 
the  soldiers  directly  involved  in  the 
learning  process.  The  officer  or  NCO  in 
charge  merely  guides  the  discussion  and 
summarizes  key  points.  During  field  evalua- 
tions of  engagement  simulation  training,  it 
quickly  becomes  apparent  that  the  most  impor- 
tant training  occurs  during  step  2,  because 
it  is  during  this  step  that  the  soldiers 
"learn"  from  their  successes  and  mistakes. 

Step  3 consists  of  successive  repetitions 
of  two-sided,  free-play  tactical  exercises 
and  afteraction  reviews.  During  this  step 
the  soldiers  have  an  opportunity  to  em;  jy 
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and  reinforce  their  new  skills,  try  out  new 
tactics  or  techniques,  and  in  turn  gain  addi- 
tional skills.  Successive  repetitions  of 
tactical  exercises  are  an  essential  element 
of  engagement  simulation  training,  exposing 
soldiers  to  the  complex  interactions  of 
ground  weapons  and  tactics.  The  techniques 
required  to  fight  and  survive  on  the  battle- 
field cannot  be  mastered  in  only  one  or  two 
exercises.  Repetition  teaches  coping  with 
lethality,  and  adds  another  important 
feature--competition.  Since  the  soldiers 
trained  quickly  perceive  that  it  is  entirely 
within  their  power  to  team  up  to  eliminate 
"enemy"  soldiers  and  accomplish  their  team 
mission  in  the  process,  they  begin  to  view 
the  training  exercises  as  a contest.  Each 
time  they  take  part  in  an  engagement  simula- 
tion exercise  they  become  more  determined  to 
defeat  the  opposing  force.  When  they  lose, 
they  pay  particular  attention  during  after- 
action reviews,  and  seek  help  from  their 
leaders  during  breaks  in  order  to  develop  the 
skills  which  will  enable  them  to  win  the 
next  time. 

At  the  current  time,  two  engagement  simu- 
lation systems  have  been  developed  and  their 
implementation  throughout  the  Army  is  nearing 
completion. 

The  first  system  is  called  SCOPES,  an 
acronym  for  Squad  Combat  Operations  Exercise, 
Simulated.  In  this  simulation,  a six  power 
telescope  is  attached  to  each  soldier's  rifle 
and  three  inch  two-digit  numbers  are  affixed 
to  his  helmet.  The  size  of  the  number  and 
the  power  of  the  telescope  were  chosen  to 
simulate  the  weapon  effect,  in  that  the 
probability  of  reading  another  soldier's 
helmet  number  through  the  telescope  at  any 
given  range,  is  close  to  the  probability  of 
hitting  the  soldier  with  live  ammunition  at 
the  same  range.  When  a soldier  can  read 
another  soldier's  number  through  his  tele- 
scope, he  fires  a blank  and  calls  out  the 
number.  The  controller  with  his  fire  team 
then  radios  it  to  the  controller  with  the  op- 
posing fire  team.  That  controller  informs 
the  soldier  wearing  the  called  number  that  he 
is  a casualty  and  requires  the  soldier  to  re- 
move his  helmet  and  remain  in  place  until  the 
termination  of  the  exercise.  SCOPES  was 
implemented  Army-wide  by  the  US  Army  Infantry 
School  in  1974. 

In  early  1974  the  US  Army  Research 
Institute  (ARI)  expanded  SCOPES  to  permit  the 
employment  of  artillery  support,  mines,  tanks, 
and  antitank  weapons  during  two-sided,  platoon 
level  exercises.  This  system,  known  as 
REAL TRAIN,  has  just  been  implemented  in  US 
Army  Europe.  Implementation  in  the  remainder 
of  the  Army  will  commence  later  this  year. 

The  next  generation  of  engagement 
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simulation  devices  is  currently  undergoing 
development.  It  involves  a higher  fidelity 
simulation  of  weapons  than  telescopes  and 
numbers,  but  being  more  complex,  it  will  be 
some  years  before  it  is  fielded.  Known  as 
the  Multiple  Integrated  Laser  Engagement 
System  (MILES),  this  approach  uses  low-power- 
eye-safe  lasers  to  simulate  the  firing  and 
effects  of  the  direct  fire  weapons  found  on 
the  modern  battlefield.  MILES  consists 
of  a series  or  family  of  laser  devices, 
which  are  being  developed  for  the  M16  rifle, 
the  full  family  of  machineguns,  the  VIPER, 
DRAGON  and  TOW  antitank  weapons,  the  main 
battle  tanks  (M60A1 , A2,  A3),  and  the  M551 
Sheridan.  Follow  on  efforts  will  expand 
MILES  into  air  defense  weapons,  helicopter 
air-ground  engagement  systems,  USAF  aircraft 
and  munitions,  and  eventually  enemy  weapons 
systems.  The  prototype  packages  consist  of 
laser  transmitters  which  simulate  the  direct 
fire  characteristics  of  the  weapon  involved, 
a laser  detector  array  which  detects  and 
decodes  incoming  laser  signals,  and  hit 
indicating  mechanisms  which  combine  audio 
and  visual  signals  to  convey  near  misses, 
hits  which  are  not  kills,  and  kills.  Each 
device  is  to  be  lightweight,  and  of  such 
size  and  shape  that  its  addition  to  the 
base  weapon  will  not  affect  the  normal 
handling,  accuracy,  or  performance  of  that 
weapon.  MILES  includes  a provision  for  a 
hierarchy  of  weapons  effects.  An  infantry- 
man, for  example,  can  "kill"  another 
infantryman  with  his  M16  laser  device, 
but  cannot  disable  a tank.  Conversely,  a 
tank  can  "kill"  not  only  another  tank  but 
also  TOW  crews  and  infantrymen.  The  key  to 
this  discrimination  is  distinct  pulse  codes 
for  each  weapon,  and  discrimination  logic 
in  each  detector. 

We  anticipate  that  when  MILES  is  fully 
fielded,  it  will  add  a previously  unknown 
element  to  the  credibility,  immediacy,  and 
effectiveness  of  casualty  assessment  on  the 
training  battlefield.  Given  the  development 
of  the  proper  software  to  support  tne  system 
(work  which  is  already  underway  by  the  Army 
Research  Institute)  MILES  has  the  potential 
to  revolutionize  military  training. 

The  foregoing  has  described  the  state- 
of-the-art  of  engagement  simulation  and 
summarized  the  theoretical  foundations  of  why 
and  how  it  should  work;  that  is,  modify 
soldier  behavior  in  the  desired  direction, 
and  foster  soldier  learning.  But  we  have 
still  left  unanswered  the  question,  "Does  it 
work  in  the  real  world?"  Our  evaluations 
over  the  past  two  years,  principally  involv- 
ing SCOPES  and  REALTRAIN,  tell  us  that  it 
does. 

The  US  Army  Training  and  Doctrine  Command 
(TRADOC)  has  just  concluded  a six-month  joint 


effort  with  US  Army  Europe  and  ARI  to  imple- 
ment REALTRAIN  in  the  divisions  in  Europe. 
While  this  effort  was  principally  directed 
toward  training  controllers,  a great  deal  of 
tactical  training  and  data  collection  took 
place  as  well.  While  not  all  of  the  data 
has  been  analyzed  or  even  examined  at  length, 
a preliminary  report  has  been  prepared  by 
ARI  which  summarizes  some  of  the  findings. 

The  implementation  effort  took  place  at 
four  different  sites,  with  three  (and  at  one 
site--four)  weeks  of  training  at  each  site. 

Two  groups  were  involved,  "A"  teams,  which 
came  and  stayed  throughout  the  entire  period, 
the  hypothesis  being  that  their  performance 
would  improve  as  the  training  weeks  passed, 
and  "B"  teams,  who  were  changed  every  week, 
and  were  thus  expected  to  be  more  inexpert 
and  serve  a control/baseline  function.  The 
relative  force  ratio  of  1:1  was  maintained 
throughout  the  exercises,  each  side  consisting 
of  a tank  platoon,  two  TOWs,  and  two  Infantry 
Squads  mounted  in  Armored-Personnel  Carriers 
(APC).  The  missions  assigned  were  limited 
to  meeting  engagements  or  attack/defense 
(delay). 


The  primary  objective  data  in  a REALTRAIN 
exercise  are  the  casualties  incurred  by  both 
sides  in  two-sided,  free-play  exercises. 

With  increases  in  tactical  proficiency  it 
would  be  expected  that  there  would  be  changes 
in  the  number  of  casualties  which  occurred. 
These  changes  could  come  about  in  two  ways. 

With  increasing  tactical  proficiency,  a 
combat  unit  should  reduce  the  number  of 
casualties  it  incurs  (by proper  use  of  cover, 
concealment,  suppression,  and  proper  movement 
techniques),  while  at  the  same  time  increas- 
ing the  number  of  casualties  it  infl (cts  on 
the  opposing  force  (by  more  effective  employ- 
ment  of  all  available  weapons). 

Meeting  Engagements:  Figures  1 and  2 show 

the  percent  of  casualties  incurred  by  Team  A 
(experienced)  and  Team  B (inexperienced)  for 
weeks  one  and  three  of  training  for  tanks 
and  infantry.  Figure  1 for  tank  casualties, 
shows  that  both  teams  sustained  approximately 
the  same  proportion  of  casualties  during  the 
first  training  week  (Team  A:  48?,  Team  B: 

45?),  as  would  be  expected  when  both  teams 
have  had  the  same  (limited)  amount  of 
tactical  training. 
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Figure  1.  Comparison  of  Tank  Casualties 
for  Team  A and  Team  B 
Meeting  Engagements  (N) 
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There  is  a large  difference  between  the 
two  teams  for  the  third  week;  Team  A sustain- 
ed 35.7%  casualties,  while  Team  B sustained 
66.7%  casualties,  thus  indicating  that 
REALTRAIN  does  increase  tactical  unit  profi- 
ciency over  time. 

From  Figure  2,  it  may  be  seen  that  durinq 
the  first  week  of  training.  Team  B lost  more 
infantrymen  than  Team  A (49%  vs  34%). 


on  expert  military  judgement  and  are  in 
general  agreement  with  weightings  found 
in  other  firepower  indices.  For  this  WC I 
a lower  value  was  better;  that  is,  a force 
sustained  fewer  casualties  itself.  The 
Weighted  Casualty  ^ndex  used  was: 

WCI  = 35  (#  tanks  immobilized  or  killed) 
+ 25  (#  TOWs  killed)  + 15  (#  APCs  killed) 

+ 1 (infantry  casualties) 
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Figure  2.  Comparison  of  Infantry 

Casualties  for  Team  A and 
Team  B Meeting  Engagements  (N) 


In  the  third  week  the  A Team  reduced  its 
casualties  slightly  (34%  to  30%),  while  the 
B Team  casualties  increased  (49%  to  55%). 

The  results  in  terms  of  infantry  casualties 
are  not  as  clear-cut  as  they  were  for  tank 
casualties,  although  there  was  a demon- 
strable performance  difference  durinq  the 
third  week  of  training  between  the  two  teams. 
By  the  third  week,  the  A Teams  were  beginning 
"to  qet  it  all  together"--reducing  the  casu- 
alties incurred  on  themselves  and  increasing 
the  casualties  inflicted  on  the  B Teams. 

The  use  of  a Weighted  Casualty  Index 
(WCI)  allows  diverse  categories  of  data  to 
be  integrated  into  a single  measure.  In 
addition,  differences  in  the  utility  of 
various  weapons  systems  can  be  taken  into 
account.  The  weightings  employed  are  based 


Figure  3 presents  training  effective- 
ness across  all  exercises  as  measured  by 
the  Weighted  Casualty  Index.  During  Week 
1,  no  difference  was  observed  between  the 
A teams  and  the  B teams,  since  the  two 
teams  had  equivalent  training  exposure,  and 
no  difference  would  be  expected. 

However,  data  from  the  second  and  third 
weeks,  comparing  increasingly  experienced 
A teams  with’ inexperienced  B teams,  show  that 
performance  differences  were  observed. 

During  the  second  week,  A teams  incurred 
smaller  losses  than  the  B teams,  and  in 
Week  3,  Team  A similarly  incurred  smaller 
losses  than  the  B teams. 

It  is  interesting  to  note  that  a 
comparison  of  the  WCI  for  the  A Team  alone 
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for  Weeks  1 and  3 shows  a significant 
improvement  in  overall  casualty  reduction. 

Indirect  Fire:  Indirect  fire  data  were 

collected  during  the  play  of  the  exercises 
for  each  team.  The  critical  data  extracted 
from  the  artillery  control  sheets  for  this 
analysis  were  the  number  of  simulated 
rounds  fired  by  each  team  which  were  then 
converted  into  the  number  of  rounds  that 
would  actually  have  been  fired. 

In  reducing  the  raw  data,  casualty 
counts  were  based  on  casualties  infl icted 
rather  than  casualties  received.  JT 
casualty  was  defined  as  a vehicle,  vehicle 
crewman,  or  infantryman  killed  or  a vehicle 
immobilized.  For  purposes  of  the  indirect 
fire  analysis,  all  casualties  were  equally 
weighted.  For  example,  an  Immobilized 
tank  was  equivalent  to  a destroyed  APC  or 
an  individual  soldier  killed.  Since  there 


was  no  way  to  directly  link  a casualty  to  a 
specific  fire  mission,  it  was  not  possible 
to  analyze  indirect  fire  usage  as  a 
function  of  type  of  casualty  inflicted. 

Equal  weighting  of  casualties  seemed  the 
most  reasonable  alternative. 

Three  summary  statistics  are  reported: 
TOTAL  CASUALTIES,  TOTAL  ROUNDS,  and  a 
CASUALTIES/ROUND  measure  of  efficiency. 

TOTAL  CASUALTIES  (which  are  casualties 
inflicted,  not  incurred)  and  TOTAL  ROUNDS 
are  found  by  summing  the  appropriate  figures 
from  all  exercises  included  in  the  scope  of 
the  table.  CASUALTIES/ROUND.  calculated 
by  dividing  the  TOTAL  CASUALTIES  by  TOTAL 
ROUNDS,  is  a general  index  of  the  efficiency 
of  the  indirect  fire  use.  Figure  4 
summarizes  the  use  of  artillery  for  both 
A teams  and  B teams  for  all  the  training 
conducted  at  all  sites  during  all  54 
exercises. 
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.089 

(N  = 54) 

B 

295  5491 

Figure  4.  Indirect  Fire  Analysis 
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Figure  7.  Weapon  Inflicting  First  Casualty  (All  Missions) 
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Overall,  Team  B fired  considerably  more 
rounds  than  Team  A,  while  inflicting  fewer 
casualties.  Again,  Team  A was  consistently 
more  effective  than  Team  B. 

In  order  to  determine  if  there  was  a 
relationship  between  which  side  initially 
detected  and/or  engaged  the  other  and  amount 
of  training,  this  analysis  looked  at  which 
team  was  able  to  detect  and  engage  the  other 
first,  how  quickly,  and  at  what  range. 

Figure  5 contains  the  number  of’ times  that 
each  team  made  the  initial  detection  and 
engagement  for  meeting  engagements.  The 
data  are  then  summed  over  all  weeks  and 
sites. 

Initial  detection  and  engagement  events 
were  split  equally  between  the  teams  in  the 
first  week.  However,  in  the  second  week, 
and  continuing  on  into  the  third,  Team  A 
clearly  enjoyed  the  advantage  in  detecting 
and  engaging  their  opponents. 

In  addition  to  noting  who  detected 
first,  the  times  and  distances  for 
these  detections  were  also  collected 
and  summarized.  Figure  6 shows  de- 
tection time  and  range  for  all  weeks 
and  all  exercises. 

Overall,  Team  A detected  the  enemy  in 
shorter  times  than  did  Team  B.  Time 
equating  to  distance,  as  it  does  in  this 
case,  this  earlier  detection  time  means 
detection  at  greater  average  distances  as 
Figure  6 illustrates.  This  greater 


distance  of  course  translates  into  more 
space  for  small  unit  commanders  to  deal  with 
the  enemy,  and  bring  into  play  sooner  the 
increased  lethality  of  all  the  weapons 
found  on  the  modern  battlefield. 

Finally,  the  data  for  weapons  inflicting 
the  first  casualties  is  displayed  in 
Figure  7. 

In  summation,  it  appears  as  if  increased 
training  resulted  in  an  increased  ability 
to  detect  and  engage  the  enemy  first,  at 
least  in  meeting  engagements.  The  time 
required  to  detect  the  enemy  was  shorter 
for  the  trained  (Team  A)  than  it  was  for 
the  untrained  (Team  B).  The  data  for 
weapons  inflicting  the  initial  casualties 
tend  to  indicate  that  artillery, 
particularly  preplanned  fires,  account  for 
the  majority  of  early  casualties. 

Tank  Losses:  The  results  presented  up 

to  this  point  have  been  related  to  training 
effectiveness.  The  data  collected  also 
shows  evidence  of  realistic  and  credible 
simulation  of  weapons  effects  during  the 
REALTRAIN  exercises,  thus  it  is  possible 
to  look  at  the  data  in  terms  of  weapons 
effectiveness. 

Figures  8 and  9 depict  the  percent 
:asualties  inf! icted  by  each  weapon 
type. 
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Tanks  51% 

TOHs  25% 

DRAGON  4%** 

90mm  & LAW  19% 

Grenade  2% 


**  DRAGON  not  played  at  all  sites. 


Figure  8.  Tank  Losses  as  Function  of  l.eapon  Type 
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Figure  9.  Tank  Losses  as  Function 
of  Weapon  Type 
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Figure  10.  Comparison  of  REALTRAIN  to  Other  Methods 
of  Collective  Training 


Training  value  of  REALTRAIN  exercises  for  a controller 
as  compared  to  participants: 

Equal  to  or  greater  than:  86% 

Less  than:  14% 

Figure  11.  Tactical  Training  Value  of  REALTRAIN 
Exercises  for  Controllers 


Figure  8 Tank  Losses  indicate  that 
across  all  sites  approximately  50%  of  the 
tanks  killed  were  killed  by  other  tanks, 

by  TOWs  and  25%  by  a combination  of 
weapons  in  the  hands  of  the  individual  foot 
soldier--the  Light  Anti-tank  Weapon  (LAW) 
90nim  recoilless  rifle  (RR),  grenades,  and 
the  DRAGON. 

Figure  9 shows  data  from  two  of  the 
training  sites  which  was  amalgamated  in 
Figure  8. 

One  of  the  things  Figure  9 illustrates 
is  that  REALTRAIN  provides  a simulation  of 
weapons  effects  and  capabilities  with  face 
validity.  In  the  large  open  area  with 
longer  fields  of  fire,  the  long  range  anti- 
tank weapons  (tank  guns  and  TOWs)  did 
nearly  90%  of  the  fank  killing.  On  the 
other  hand,  when  visibility  and  fields  of 
fire  became  restricted  in  the  smaller  area, 
nearly  half  of  the  tank  kills  were  with 
short  range  infantry  weapons,  LAW,  90mm  RR, 
and  the  grenade. 

No  discussion  of  engagement  simulation 
can  be  considered  complete  without  some 
discussion  of  the  acid  test  of  any  training 
system--troop  acceptance.  For  a system  to 
be  successful  the  soldier  must  believe  that 
he  can  use  it,  with  confidence,  and  that 
its  use  will  benefit  him.  During  post- 
exercise interviews  during  the  USAREUR 
REALTRAIN  implementation,  soldiers,  leaders, 
and  controllers  were  polled  on  their 
perceptions  about  engagement  simulation 
training. 

A Leader-Control ler  Questionnaire  was 
administered  to  34  3 controllers  and  38 
leaders  ranging  in  grades  from  E-4  to  0-3. 
Most  respondents  were  in  grades  E-6,  E-7, 

0-1  and  0-2,  representing  primarily  squad 
and  platoon  level  NCOs  and  officers. 

The  questionnaire  sought  to  obtain  data 
about  recent  unit  training  experiences 
(other  than  REALTRAIN)  as  well  as  reactions 
to  REALTRAIN  and,  for  controllers,  their 
reaction  to  their  experience  as  controllers. 
Responses  were  typically  quite  favorable 
to  REALTRAIN.  As  Figure  10 
shows,  REALTRAIN  was  reported  as  more 
effective  than  live  fire  exercises  and 
traditional  field  exercises. 

Controllers  were  also  asked  about  their 
experiences  as  controllers.  Responses  in 
Figure  11  demonstrate  that  86%  of 
controllers  feel  that  the  tactical  train- 
ing value  of  REALTRAIN  exercises  while 
serving  as  controllers  was  equal  to  or 
greater  than  the  value  for  a participant. 


Figure  12  indicates  that  90%  of  the 
controllers  and  leaders  considered 
REALTRAIN  "very  effective"  for  training 
in  use  of  terrain  for  cover  and  conceal- 
ment, 73%  felt  it  was  "very  effective" 
in  training  on  employment  of  all  available 
weapons,  while  62%  felt  it  was  "very 
effective"  in  training  on  employment  of 
indirect  fire. 

Finally,  we  turn  to  the  soldiers  them- 
selves, those  who  participated  in  the 
tactical  exercises.  Their  acceptance  was 
overwhelmi ng. 

A questionnaire  was  completed  by  542 
soldier  participants;  302  with  an  infantry 
M0S  (56%)  and  240  with  an  armor  M0S  (44%). 
The  response  to  the  questionnaire  shows  no 
systematic  or  significant  differences  in 
responses  among  sites,  teams,  armor  versus 
infantry,  or  rank  of  respondent.  Therefore, 
the  results  presented  in  Figure  13 
discuss  the  results  of  all  participants 
without  any  further  breakdowns. 

The  responses  to  a question  on  the 
perceived  state  of  training  before  and 
after  participation  in  REALTRAIN  are 
summarized  in  Figure  13.  The  results  speak 
for  themselves! 

Figure  13  shows  graphically  that  prior 
to  REALTRAIN  only  43%  felt  that  they  were 
adequately  trained,  but  that  after 
exposure  to  REALTRAIN  training,  86%  felt 
that  they  were  adequately  trained. 

Probably  underlying  the  perceived 
improvement  in  state-of-unit  training  is 
the  fact  that  most  participants  felt  that 
REALTRAIN  provided  more  effective  training 
than  their  normal  unit  training. 

When  the  soldiers  talked  about 
REALTRAIN,  their  universal  acceptance  was 
really  evident.  Said  one  sergeant: 

"You  can  see  it.  In  REALTRAIN  you 
see  your  mistakes  and  talk  about  them. 
Why  did  I get  killed?  Why  did  that 
TOW  see  me?  How  did  this  infantryman 
get  on  top  of  my  track  and  throw  a 
grenade  inside?  You  can  see  it  and 
you  know  your  mistakes,  and  very 
rarely  do  you  make  the  same  one  twice." 
Staff  Sgt,  Co  B,  3d  Bn,  32  Armor 
(A  Team) 

An  infantry  squad  leader  commented  on 
the  realism  and  utility  of  the  REALTRAIN 
engagement  simulation  system: 
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How  effective  do  you  consider  REALTRAIN 
to  be  for  training  units  to: 
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a. 
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9? 
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b. 

Employ  indirect  fire 

62? 

35% 

3% 

c. 
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availabl  e weapons 

73? 

25% 

2% 

Figure  12.  Effectiveness  of  REALTRAIN  for  Tactical  Training 
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Figure  13.  Troop  Perceptions  of  State  of  Unit  Training 
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"I  am  a combat  vet.  Most  of  the  guys 
out  here  didn't  have  any  idea  about  it, 
not  in  my  squad.  But  right  now  they 
know  what's  involved  and  they  are 
keeping  their  heads  down,  their  tails 
down,  they're  moving  fast  and  they're 
moving  right.  They  are  working  as 
teams . " 

Concerning  combined  arms  tactics  this 
same  squad  leader  said: 

"Killing  tanks.’  You  want  to  get  the 
tanks.  We  found  out,  the  tankers 
have  found  out,  that  we  infantry 
can  get  those  tanks.  Once  we  can 
get  those  tanks  in  the  woods  we  can 
tear  them  up;  they  can  get  us  in 
open  terrain,  that's  their  meat. 

Staff  Sat,  Co  C,  1st  Bn,  36th  Inf 
(Team  A)." 


Perhaps,  though,  the  most  telling  and 
eloquent  comment  of  all  is  this  one  from 
an  armor  company  platoon  sergeant,  after 
3'j  weeks  of  training. 


"I  say  it’s  the  best  training  I've 
seen  since  I have  been  in  the  Army 
--it  just  can't  be  compared  with 
anything  else.  My  men  want  to 
stay  another  month.  Since  we 
began  training,  I've  had  a total 
of  three  sick  calls  and  those 
were  because  of  the  flu  epidemic. 
My  appointments  are  just 
emergency  type  things.  Account- 
ability and  morale  is  100  percent. 
It  has  been  beautiful . " 
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SUMMARY 

This  paper  presents  a method  of  simu- 
lating various  noise  sources  in  a Weapons 
Fire  Simulator  System  which  has  been  mod- 
eled as  an  Optical  Cormunications  Channel. 
This  Weapons  Fire  Simulator  System  is  com- 
posed of  laser  transmitters  mounted  on 
weapons  that  fire  blank  cartridges,  and 
laser  receivers  mounted  on  targets.  The 
laser  transmitter  sends  out  "kill"  beam 
pulses  to  the  target  whenever  blank  car- 
tridges are  fired.  Detection  of  these 
pulses  at  the  target  signifies  a “hit."  The 
entire  system  along  with  the  optical  commu- 
nication channel  is  simulated  in  a general- 
purpose  computer  program  called  SCEPTRE. 

This  analysis  package  is  an  efficient  means 
of  modeling  the  communication  channel  char- 
acteristics and  determining  signal-to-noise 
ratios  as  functions  of  various  electrical 
and  physical  parameters.  Also  the  SCEPTRE 
program  is  a versatile  tool  for  circuit 
noise  calculations.  The  main  advantage  is 
a single  SCEPTRE  run  computes  the  total 
noise  output  from  a large  number  of  noise 
sources  distributed  throughout  the  circuit. 


INTRODUCTION 


The  computer-aided  design  package 
SCEPTRE  is  used  to  simulate  a portion  of  a 
Weapons  Fire  Simulator  System. 

SCEPTRE  is  an  automatic  circuit  anal- 
ysis program  that  solves  for  the  transient 
and  steady  state  response  of  large  networks. 
SCEPTRE  solves  circuit  equations  from  a 
description  of  circuit  topology  and  compo- 
nent values.  In  addition,  a defined  para- 
meter mode  solves  state  equations  which  can 
describe  any  system.  This  approach  models 
the  receiver  section.  The  communication 
channel  is  modeled  using  FORTRAN  IV  sub- 
routines. SCEPTRE's  circuit  description 
is  composed  of  elements,  defined  parameters, 
and  functions.  Element  values  can  be  de- 
fined by  evoking  FORTRAN  IV  subroutines. 
Subroutine  parameters  can  be  either  circuit 
variables,  defined  parameters,  or  constants. 
The  subroutine  facility  is  used  to  specify 
noise  sources  and  atmospheric  effects  on 
signal  sources.  Any  of  these  parameters 
can  be  changed  via  the  rerun  feature. 
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A block  diagram  of  the  laser  optical 
communication  system  representing  the  wea- 
pon fire  simulator  is  shown  in  Figure  1. 

The  laser  transmitter  emits  a digital  pulse 
code  whenever  the  weapon  is  fired  which  in- 
cludes weapon  identifying  data.  The  laser 
beam  passes  thru  an  optics  system  into  the 
communication  channel , the  atmosphere.  The 
channel  characteristics  include  atmospheric 
turbulence,  scattering,  and  absorption. 
These  effects  cause  the  laser  beam  to  di- 
verge, bend,  attenuate,  and  scintillate. 
These  effects  are  a function  of  the  path 
length  and  specific  meteorological  condi- 
tions. The  direct  or  indirect  sunlight  il- 
luminating the  laser  detectors,  cause  a 
constant  dc  bias  in  the  photodiodes  genera- 
ting shot  noise.  The  signal  amplifier  gen- 
erates shot  and  thermal  noise.  The  output 
of  the  signal  amplif^r  feeds  digital 
processing  circuitry. 


The  electrical  noise  sources  in  passive 
and  active  electronic  components  have  been 
extensively  examined  for  many  years.  *5 
There  is  an  increasing  need  for  computer- 
aided  noise  analysis  since  calculations 
associated  with  noise  in  electronic  circuits 
are  often  extremely  tedious  and  complicated. 
In  these  calculations,  noise  is  superimposed 
on  the  electronic  circuit  as  a current  or 
voltage  source  whose  value  follows  the 
normal  distribution  probability  curve  with 
its  mean  value  zero  and  standard  deviation 
a function  of  various  parameters. 

The  SCEPTRE  program  is  a general  cir- 
cuit analysis  program  having  many  active 
device  models  available  in  its  "library." 

One  can  add  the  noise  sources  into  tht  ex- 
isting SCEPTRE  device  models  for  calcula- 
tions, or  the  user  can  supply  his  own  model 
and  do  the  same. 

Noise  Sources 

The  sources  of  noise  encountered  are 
shot  noise,  thermal  noise,  and  excess  noise. 
The  magnitude  of  the  shot  noise,  having  a 
constant  spectral  oensity,  can  be  calculated 
from  physical  parameters.  We  call  this  type 
of  noise  a white  noise  source.  Shot  noise 
is  due  to  the  fluctuations  of  the  number  of 
charge  carriers  flowing  through  a junction 
barrier  in  a solid  state  device.  For  a for- 
ward biased  PN  junction  photodiode,  the 
spectral  density  of  the  current  fluctuation 
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has  an  expression  Si(f)=2el  where  "e"  is  the 
electron  charge  in  coulombs,  and  I is  the  dc 
current  flowing  through  the  photodiode.  The 
thermal  noise  is  caused  by  the  random  vibra- 
tion of  the  charge  carriers  in  a resistor  due 
to  thermal  agitation.  Over  the  practical  fre- 
quency range  of  a circuit's  operation,  the 
thermal  noise  has  a flat  spectral  density  and 
is  considered  to  be  a white  noise  source. 

The  excess  noise  most  often  encountered  in 
electronic  circuits  is  the  1/f  type  which  has 
a spectral  density  of  the  form  S(f)=K/f^,  where 
K is  a constant  and  "b"  has  the  value  close 
to  unity.  The  other  type  of  excess  noise 
found  in  semiconductors  and  photodiodes  is 
burst  noise.  This  noise  consists  typically 
of  random  pulses  of  variable  length  and  equal 
height.  Burst  noise  has  a power  spectral 
density  varying  with  frequency  in  the  form  of 
S(f)=K/fa  but  with  "a"  approximately  2. 

Since  both  types  of  excess  noise  have  spec- 
tral densities  increasing  as  the  frequency 
decreases,  they  are  dominant  noise  components 
only  in  the  low  frequency  region  (below  10 
KHz).  This  analysis  deals  with  high  fre- 
quencies masking  the  excess  noise  with  shot 
and  thermal  noise.  Assume  excess  noise  is 
negligible  and  equal  to  zero  for  this  paper.2 

The  noise  sources  in  electronic  circuits 
are  normally  represented  in  the  form  of  vol- 
tage and  current  sources.  The  above  noise 
sources  are  two  types;  one  with  the  spectral 
density  independent  of  frequency  (called 
white  noise),  and  the  other  with  the  spectral 
density  varying  with  frequency  (called  col- 
ored noise). 

Simulation  of  Noise  Sources 


Both  white  and  colored  noise  sources 
can  be  simulated  by  insertion  of  FORTRAN  IV 
subroutines  into  the  SCEPTRE  source  deck. 

The  SCEPTRE  program  is  well  known  for  its 
simplicity  and  capability  for  handling  the 
input  and  output  signals  of  a circuit  with 
time  as  the  independent  variable.  This  cap- 
ability of  the  program  makes  it  especially 
efficient  for  circuit  noise  problems  because 
all  the  noise  sources  in  a circuit  can  be 
simulated  with  current  or  voltage  noise  sour- 
ces which  have  the  required  spectral  inten- 
sity. For  this  paper,  the  FORTRAN  IV  sub- 
routine GAUSS  which  is  available  in  the  IBM/ 
360  Scientific  Subroutine  Package  was  used  to 
simulate  the  white  noise  sources.  The  simu- 
lation was  achieved  by  using  the  normally 
or  Gaussian  distributed  random  numbers  gen- 
erated by  the  subroutine  GAUSS  as  a voltage 
or  current  source  in  the  circuit.  To  use 
this  subroutine,  the  me^n  value  and  standard 
deviation  of  the  distribution  must  be  given 
in  the  program.  Set  the  mean  value  of  the 
noise  to  zero  since  we  are  interested  only 
in  the  ac  component  of  the  noise.  The  stand- 
ard deviation  of  the  shot  noise  distribution 


can  be  determined  via  the  following.  Let 
the  Lime  interval  between  the  neighboring 
randomly  generated  numbers  be  a and,  if  the 
number  of  the  random  points  generated  is 
large  enough,  the  spectral  intensity  of  the 
noise  source  will  have  an  expression 


Sv(f ) 


Sw  ;0 


0;  elsewhere 


where  Sw  is  the  spectral  intensity  of  the 
white  noise  source.  For  this  analysis,  only 
the  positive  frequency  is  considered.  The 
mean  square  magnitude  or  the  variance  of 
this  noise  can  be  represented  as 

V2  = 3.204  x 10"19  If 
The  standard  deviation  is 

a =s/Hr =y  3.204  x lO’19  If 


where  I is  the  photo  diode  current  generator 
JA,  or  JB  thru  JO  in  Figure  3.  The  time 
interval  a is  determined  by  the  highest  fre- 
quency of  interest  with  the  maximum  fre- 
quency of  interest  being 

f = 1 

2a 

For  example;  to  generate  a shot  noise  source 
which  has  a constant  spectral  intensity  from 
dc  up  to  10  MHz, 

a = 1 = 1 7 = 50x  10_9  sec 

27  2 x 107 


should  be  used.  It  is  important  to  note 
that  all  the  noise  sources  simulated  in  this 
way  are  uncorrelated,  since  each  photo  diode 
has  its  own  GAUSS  subroutine,  and  contain 
all  the  noise  power  in  the  usable  circuit 
bandwidth.  These  subroutines  can  be  simul- 
taneously inserted  into  the  SCEPTRE  program 
to  compute  the  instantaneous  values  of  the 
noise  at  the  output  of  the  circuit  being- 
simulated.  From  these  values  of  noise,  the 
noise  power  can  be  easily  calculated  by 
another  FORTRAN  IV  subroutine  that  performs 
these  calculations. 

The  thermal  and  shot  noise  in  the 
signal  amplifier  are  simulated  by  using 
the  signal  amplifier's  measured  output  noise 
voltage.  Divide  this  value  by  the  amplifi- 
ers gain  and  use  as  the  standard  deviation 
for  the  normal  distribution.  The  mean  value 
is  set  to  zero.  Assume  a noise  bandwidth  of 
10  MHz , then 

A = _L  = _L  ,=50  x 10*9  sec 
2f  2 x 107 

Transmitter  and  Receiver  Efficiency  Factors 

These  factors  are  due  to  the  optics 
at  the  laser  transmitter  and  receiver.  The 
transmitter  efficiency  factors  lower  the 
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effective  laser  output  power.  For  a typical 
laser  transmitter  the  transmitting  efficiency 
is  22 % with  the  receiver  efficiency  30%. 

Thus,  the  equivalent  laser  power  at  the  re- 
ceiver's detectors  is  a function  of  the  pro- 
duct of  the  transmitter  and  receiver  effici- 
ency factors. 

Simulation  of  Atmospheric  Effects 

For  the  purpose  of  laser  system  design, 
atmospheric  effects  may  be  classified  as  at- 
tenuation and  turbulence.  For  the  design  of 
a laser  for  an  optical  coirmunication  link, 
where  the  target  or  receiver  is  smaller  than 
the  laser  beam,  both  attenuation  and  turbu- 
lence effects  must  be  considered.3  Attenu- 
ation in  the  atmosphere  is  caused  by  the 
mechanism  of  absorption  and  scattering.  Most 
lasers  have  output  wavelengths  outside  the 
absorption  bands  which  are  selected  sharp 
bands  throughout  the  electromagnetic  spec- 
trum. Scattering  attenuation  of  a laser  beam 
occurs  when  the  photons  in  the  laser  beam  are 
deflected  by  particles  in  the  air.  At  short- 
er wavelengths,  the  air  molecules  will  cause 
scattering  attenuation,  but  at  longer  wave- 
lengths, microscopic  particles  of  dust  and 
smoke  are  the  primary  scattering  media.  At 
0.904  microns,  attenuation  is  mostly  due  to 
particle  scattering.  Visibility  is  also  a 
function  of  atmospheric  attenuation  and 
serves  as  a convenient  parameter  for  speci- 
fying the  attenuation.  Turbulence  effects 
are  caused  by  refractive  index  inhomogenei- 
ties in  the  atmosphere  and  may  result  in 
deflection  or  spreading  of  the  laser  beam 
or  in  nonuniform  energy  distribution  across 
the  wavefront.  All  of  these  effects  cause 
degradation  of  performance,  resulting  in  re- 
duced energy  density  on  the  target  or  recei- 
ver. 


Atmospheric  Attenuation 

A FORTRAN  IV  subroutine  calculates  the 
percent  transmission  through  the  atmosphere 
of  the  laser  beam  based  upon  the  path  length, 
and  wavelength.  The  transmission  of  the 
laser  beam  through  the  atmosphere  is  given 

by 

T = e-  aR 

where  a is  the  attenuation  coefficient,  and 
R the  path  length.  This  coefficient  is  a 
function  of  molecular  scattering,  particle 
scattering,  and  absorption.  For  a given  wave- 
length, a is  a sum  of  these  factors.  Visi- 
bility is  defined  as  the  range  where  a target 
f 100J  contrast  is  seen  as  a target  wiu,  its 
-parent  contrast  reduced  to  2%.  An  accurate 
■«’i  ’ri  .al  relationship  between  a and  the  vis- 
■ty  accounting  for  the  variation  of  a 
. 1 f wavelength  i s 

. 3^91  |0.55j  g 


where  "g"  is  a function  of  visibility  (Rv) 
which  varies  according  to  the  following 


Rv(km) 

■0-6 

6-9 

9-12 

12-100 


(Rv/5)V3 

(0.86+Rv/30) 

(0.98+Rv/30) 

(1.15+RV/200) 


Atmospheric  Turbulence 

The  atmosphere  is  an  inhomogeneous 
medium  made  up  of  regions  of  varying  size 
in  which  the  refractive  index  deviates  from 
the  mean  value.  When  a laser  beam  passes 
through  the  atmosphere,  these  regions  result 
in  several  effects.  These  effects  are  beam 
steering  (the  entire  beam  is  deflected  from 
its  original  path),  spreading  (the  divergence 
of  the  beam  is  increased),  and  scintillation 
(phase  changes  across  the  wavefront  result 
in  constructive  and  destructive  interference 
causing  amplitude  changes  in  the  wavefront 
that  vary  in  a randomly  distributed  Gaussian 
fashion) . 


Beam  steering  is  assumed  negligible, 
and  beam  divergence  is  a nonprobabil istic 
function  for  this  analysis.  Let  the  laser 
output  power  be  PWR,  XDIV  the  beam  diver- 
gence in  the  X direction,  and  YDIV  the  beam 
divergence  in  the  Y direction.  The  fraction 
of  the  transmitted  energy  at  the  receiver 
(PRW)  isgiven  below:g 

A = (2.54  x 10-2  + XDIV*  R)/2 
B = (2.54  x IQ'2  + YDIV*  R)/2 


pru  = PWR  (2.54  x lO-2)2  EFA 
it  (2.54)2  AB  ' 

PWR  = POWER  * XET  * XER 

XET  = Efficiency  factor  for  transmitter 

XER  = Efficiency  factor  for  receiver 


where  R is  the  path  length,  Xd  the  X distance 
from  the  laser  beam  center  to  the  target 
center,  and  Yd  the  Y distance.  PRW  is  the 
laser  beam  power  at  the  detector  after  beam 
divergence  and  efficiency  factors  have  been 
applied  to  the  transmitter  and  receiver. 

PWR  is  the  laser  power  output  after  effi- 
ciency factors  have  been  applied.  See 
Figure  2 for  an  illustration. 


Scintillation  Effects 


A FORTRAN  IV  subroutine  calculates  the 
scintillation  effect  simulated  by  the  sub- 
routine GAUSS.  Tatarski's  work  has  shown 
that  the  intensities  in  a laser  beam  cross 
section  are  log  normally  distributed  (have 
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BACKGROUND  SHOT  IS  TO  A/D 


Block  diagram  of  weapon  fire  simulator 
viewed  as  an  optical  communication  system 


Figure  1. 
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Beam 

Center 


/-Intensity 


Cross-sectional  view  of  laser  beam  Intensity  distribution. 

The  beam's  elliptical  cross  section  Is  defined  by  the  threihold  value, 
the  peak  power  and  the  beam  divergence  of  the  transmitter. 


Figure  2. 
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a normal  distribution  when  plotted  with  the 
intensity  on  a logarithmic  scale),  with  the 
variance  of  the  GAUSSIAN  distribution  being 

o?  = 1.23  Cl  K 7/6  R11/6 
t n 

for  an  infinite  plane  wave  case.7  Here  K is 
the  wave  number  2n/x  corresponding  to  the 
wavelength  being  propogated,  Cn  the  turbu- 
lence structure  constant  of  the  atmosphere, 
and  R the  path  length.  The  three  classes  of 
turbulence  are. 

Weak  turbulence  Cn  = 8 x 10“^  m “1/3 


Intermediate 

turbulence  Cn  = 4 x 10'°  m *1/3 

Strong  turbulence  Cn  = 5 x 10"7  m "1/3 


During  different  times  of  day  and  seasons  of 
the  year,  use  different  turbulence  con- 
stants. During  the  winter  season,  use  weak 
turbulence  at  sunrise  and  intermediate  tur- 
bulence at  noon;  during  the  summer  season, 
intermediate  turbulence  at  sunrise  and 
strong  turbulence  at  noon. 

One  problem  with  the  variance  equation 
is  that  it  predicts  an  infinite  increase  of 
energy  variance  with  respect  to  path  length, 
which  is  not  the  case.  One  must  assume  that 
o.  will  saturate  at  some  level  a function  of 
both  path  length  and  . The  corrected 
value  of  variance  is 

Og  = In  (2  - exp(-o^)) 

which  saturates  at  0.6931,  the  natural  log 
of  2.  This  equation  is  valid  for  a spher- 
ical wave  but  another  correction  is  neces- 
sary for  a plane  wave.  In  our  case,  with  the 
beam  divergence  high  ()>  3 milliradians) 
assume  the  laser  is  emitting  a spherical 
wave  with  no  correction  necessary. 

The  mean  value  of  the  normal  distribu- 
tion that  SCEPTRE  generates  is 

mv=ln  (PRW*20Q*lCf 9*e“aa2/2) 

where  200X10'”=laser  pulse  length 

The  frequency  of  the  amplitude  modula- 
tion due  to  scintillation  is  determined  by 
eddy  wind  currents  in  the  atmosphere.  This 
movement  of  air  normal  to  the  path  of  prop- 
agation causes  amplitude  modulation  of  the 
scintillation  pattern.  The  maximum  fre- 
quency of  this  modulation  is  determined  by 
the  following  equation 

f = (ITS T172 

where  Vn  is  the  component  of  wind  velocity 
normal  to  the  line  of  sight,  X the  wave- 
length, and  R the  path  length.  This  maximum 
frequency  is  a measure  of  the  rate  that  tur- 
bulence statistics  change  as  the  atmosphere 


moves  across  the  laser  beam  path.  This 
serves  to  establish  the  validity  that  each 
transmitted  pulse  is  an  independent  event. 
For  our  analysis,  the  time  interval  between 
random  number  generation  is  1/f,  the  rate 
at  which  new  random  numbers  are  generated  by 
GAUSS  during  the  SCEPTRE  run. 

Example 


An  example  is  given  here  illustrating 
the  method  of  incorporating  the  simulated 
noise  sources  in  the  SCEPTRE  program  to 
compute  the  noise  power  at  the  circuit's 
output,  a function  of  the  parameters  de- 
scribed above.  The  circuit  to  be  analyzed 
consists  of  four*»olar  cells  in  parallel 
driving  a signal  amplifier,  see  Figure  3. 

The  circuitry  loading  the  solar  cells  is 
ac  coupled  and  loaded  with  an  inductor  to 
pass  only  the  received  pulses  (ac  component) 
of  the  signal.  This  is  necessary  when  the 
detectors  are  in  sunlight  generating  a dc 
bias.  The  shot  noise  in  each  solar  cell  is 
simulated  with  noise  current  generators  JNOA 
thru  JNOD.  respectively.  The  dependent  cur- 
rent generators  JA  thru  JD  simulate  the 
diode  effect  occurring  in  the  solar  cells 
due  to  their  silicon  PN  junction.  Jl  thru 
J4  simulate  the  dc  bias  introduced  by  the 
sunlight,  CJ1  thru  CJ4  the  diode  junction 
capacitances,  RSI  thru  RS4  the  internal 
series  bulk  resistances,  and  RP1  thru  RP4 
the  reverse  bias  series  resistances.  The 
signal  amplifier  is  simulated  by  a voltage 
source  and  a resistor  capacitor  load  limit- 
ing the  bandwidth.  Four  different  random 
number  sequences  are  generated  by  four 
separate  GAUSS  routines  giving  completely 
uncorrelated  shot  noise  results.  The  ENQA 
voltage  source  simulates  the  thermal  and 
shot  noise  of  the  signal  amplifier.  JPS  and 
JPT  are  the  current  sources  simulating  the 
currents  due  to  the  received  laser  pulses. 

For  this  analysis,  assume  a configuration 
with  all  four  detectors  in  full  sunlight, 
but  only  two  of  them  receiving  the  laser 
pulses.  Resistor  RT  simulates  the  output 
impedance  of  the  signal  amplifier  with  Cl 
and  RLF  a low-pass  filter  simulating  the 
actual  bandwidth  of  the  amplifier.  The  RC 
time  constant  gives  a cutoff  frequency  of 
approximately  1 MHz. 

One  computer  run  with  SCEPTRE  and  the 
FORTRAN  IV  subroutines  calculates  the  noise 
power  at  the  output  with  JPS  and  JPT  set 
equal  to  zero  (no  signal).  Another  computer 
run  calculates  the  average  peak  signal  by  set- 
ting JNOA  thru  JNOD  and  ENOA  to  zero.  These 
runs  are  compared  and  the  signal  to  noise 
ratio  determined.  The  various  parameters 
discussed  in  this  paper  can  be  varied  from 
run  to  run  with  the  signal  to  noise  ratio 
determined  as  a function  of  these  parameters. 
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This  is  the  equivalent  circuit  of  four  paral- 
lel solar  cell  photodiode  outputs  applied  to 
an  amplifier  through  a filter.  The  following 
Is  a description  of  the  circuit  parameters: 

• Transmitter  laser  power;  5 watts 

• Laser  pulse  width;  200  nanoseconds 

• Laser  signal  current  In  photodiodes;  JPS.JPT 
Due  to  scintillation,  the  laser  signal  has  a 
statistical  variation  and  for  both  JPS.JPT; 
mean  value  * m„  standard  deviation  * a, 

• Current  generated  due  to  background  Illumi- 
nation; J1,J2,03,J4 

• Current  through  PN  junction  of  solar  cell 
photodiode;  JA.JB.JC.JD 

• Shot  noise  current  generated  due  to  current 
through  PN  junction;  JN0A,JN0B,JN0C,JN0D 
mean  value  of  noises  standard  deviations 


Photodiode  junction  capacitance;  CJ1.CJ2, 
CJ3,CJ4=3nf 

Reverse  biased  leakage  res i stance ;RP1 ,RP2, 
RP3,RP4=2Kfl 

Series  resistance  due  to  Intrinsic  bulk 
material  of  the  photodiode  and  the  circuit 
loads;  RSI ,RS2,RS3,RS4=14fi 
Damping  resistance  for  control  of  transient 
response:  RL=lKfl 

Inductance  to  reduce  dc  current  through  pN 
Junction  of  photodiode;  Ll*3mh 
Resistance  of  tnductance;  RC=0.5n 
Coupling  capacitor  CX«.2yf 
Load  resistance;  RLL*200n 
Amplified  signal  voltage;  El «1 00VRLL 
Noise  voltage  in  amplifier;  ENOA 
mean  values  standard  dev1ation*2.7volts 
Internal  series  resistance  of  amplifier; 
RT-ln 

Parallel  capacitance  and  resistance  used 
to  simulate  the  amplifier  bandwidth; 

Cl-luf  RLF«200fi 


Figure  3. 
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For  a 5-watt  laser,  the  results  of 
these  runs  are  shown  in  Figure  4.  Sun 
currents  J1  through  J4  are  functions  of  the 
receiver  optics,  visibility,  and  orientation 
of  the  detectors  with  respect  to  the  sun. 
Assume  one  orientation  for  all  computer  runs. 
The  laser  beam  is  attenuated  by  the  trans- 
mitter and  receiver  efficiency  factors,  and 
subject  to  the  atmospheric  channel  effects. 
The  beam  irradiates  two  of  four  detectors. 

The  signal -to-noise  ratios  plotted  represent 
peak  current  to  rms  noise  current  values. 

No  consideration  has  been  taken  as  to  the 
effect  of  visibility  on  sun  current. 


Pulse  detection  occurs  if  the  peak 
value  of  the  signal  current  generated  by  the 
laser  pulse  exceeds  a predetermined  value 
or  threshold.  The  threshold  setting  also 
determines  how  often  a noise  spike  will  be 
incorrectly  interpreted  as  a pulse.  The 
probability  of  this  occurring  is  called  the 
probability  of  a false  alarm.  The  higher 
the  threshold,  the  less  likely  a false  alarm, 
but  also  less  likely  is  the  probability  of  a 
pulse  being  detected.  Figure  5 shows  the 
probability  of  detection,  Pd,  versus  sun 
current,  range,  visibility  and  threshold 
level.  The  threshold  to  noise  ratio,  T/N, 
can  be  set  to  any  desired  value,  and  de- 
termines the  probability  of  false  alarm, 

Pfa.3 


DISCUSSION 

The  circuit  considered  in  this  example 
was  four  parallel  solar  cells  connected  to 
an  amplifier.  The  output  of  this  amplifier 
is  applied  to  a threshold  detector  and  sent 
to  a digital  decoder.  This  paper  was  a 
study  of  only  the  communication  channel  and 
the  / detectors , hence,  an  ideal  threshold 
detector  has  been  assumed.  The  only  system 
degradation  considered  is  the  scintillation 
^f  the  atmosphere  through  which  the  signal 
traveled  and  the  additive  shot  noise  gener- 
ated in  the  solar  cell  due  to  background  sun 
light.  A typical  noise  value  was  used  for 
the  single  stage  amplifier.  For  the  usual 
applications  of  solar  cells  one  would  expect 
significantly  higher  values  of  sun  generated 
current,  but  for  this  application  this  cur- 
rent is  assumed  limited  by  a passband  opti- 
cal filter  centered  upon  the  laser  wavelength 
0. 904um. 


Figure  4 shows  the  variation  of  the 
average  signal-to-noise  ratio  for  various  a- 
mounts  of  sun  current  generated  shot  noise. 

An  average  signal  value  was  obtained  by  cal- 
culating the  amplifier  peak  signal  output 
when  a pulse  from  a 5 watt  laser  was  trans- 
mitted through  a scintillating,  attenuating 
channel,  and  then  averaging  the  peak  values 
over  a large  number  of  separate  pulses. 

Each  detector  was  treated  statistically  inde- 
pendent for  both  the  scintillation  effects 
and  the  noise.  The  noise  values  were  ob- 
tained by  calculating  the  rms  value  of  the 
amplifier  output  with  no  signal  applied.  The 
average  signal-to-noise  ratio  remains  very 
high  even  for  lower  visibility  conditions.  It 
is  only  under  the  most  sever  visibility  con- 
dition at  the  longer  ranges  does  the  noise 
and  scintillation  give  unacceptable  signal  to 
noise  ratios.  For  the  most  part,  the  ratio 
is  more  than  acceptable. 

The  graphs  in  Figure  5 illustrate  how 
the  probability  of  detection  varies  with 
range  and  visibility  conditions.  These 
graphs  also  reflect  the  high  signal  to  noise 
ratio.  The  probability  of  detection  remains 
at  almost  100%  except  for  the  most  adverse 
conditions.  This  high-detection  probability 
would  increase  further  if  additional  solar 
cells  were  illuminated  by  the  same  pulse.  The 
additional  cells  must  be  spaced  sufficiently 
far  apart  (usually  greater  than  8 cm)  so  that 
they  can  be  treated  as  statistically  indepen- 
dent. The  probability  of  signal  detection 
can  be  increased  even  more  by  the  use  of 
coding  for  single  and/or  multiple  pulse  error 
correction.  Because  of  the  very  favorable 
signal-to-noise  ratio,  the  threshold  value  can 
be  set  high  which  in  turn  makes  a false  alarm 
very  unlikely. 

CONCLUSION 

The  SCEPTRE  package  provides  the  ability 
to  describe  analytical ly,  through  tne  use  of 
FORTRAN  IV  subroutines,  the  effects  of  an 
atmospheric  communication  channel  on  laser 
transmitted  signals.  It  also  provides  ease 
in  the  modeling  of  the  laser  signal  detectors 
including  the  various  types  of  noise  which 
occur.  The  results  of  the  simulation  of  the 
channel  and  the  sensor  electronics  suggest 
that  a number  of  inexpensive  solar  ceils  can 
be  connected  to  a single  pre-amplifier  and 
still  provide  viable  signal  to  noise  charac- 
teristics. 
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The  above  curves  show  the  variation  of  average  peak  signal  to  noise  ratio  as  a function 
of  visibility  and  range.  For  all  the  above  cases  the  scintillation  value  was  taken  to 
be  the  worst  case  saturated  value  Ln  2,  for  a divergent  laser  beam. 

Figure  4. 
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Pd  Sun  current  = 15  ma  pd  Sun  current  = 20  ma  PH  Sun  current  = 25  ma 
T/N  « 15  db  T/N  = 15  db  T/N  = 15  db 
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The  above  graphs  give  the  probability  of  detection,  Pd,  of  a single  pulse  as  a function 
of  range.  Three  visibility  conditions  are  shown,  the  saturated  scintillation  condition 
Is  used  for  all  cases.  Reading  from  left  to  right,  the  sun  background  generated  Increases 
while  the  threshold  to  noise  ratio  Is  held  constant.  From  top  to  bottom  the  sun  current 
Is  fixed  while  the  threshold  to  noise  ratio  Is  Increased. 


Figure  5. 
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TANK  DRIVER  AND  TANK  GUNNER  TRAINING  SIMULATORS 


JEAN  BARADAT 
LMT  Simulators  - France 


INTRODUCTION 

We  are  witnessing  a growing  interest  from 
utilizers  in  classroom  training  for  tank  crews. 
Simulators  exist,  or  are  being  developed, 
which  provide  efficient  training.  The  purpose 
of  this  paper  is  to  specify  the  concept  of 
tank  crew  training  and  to  isolate  the  functions 
of  each  new  member,  in  order  to  better  define 
the  features  of  the  simulators  to  be  used  for 
their  training. 

Analysis  of  the  interworking  of  the 
members  of  a tank  crew  shows  that  coordination 
is  provided  entirely  by  the  tank  commander; 
there  is  practically  no  direct  relationship 
between  the  gunner  and  the  driver.  It  is 
true  that  their  actions  are  coordinated  - re- 
ducing speed,  or  taking  up  a suitable  firing 
position  - but  the  information,  especially 
visual,  which  they  individually  use  and  the 
actions  they  undertake  are  either  totally 
independent  or  only  slightly  correlated.  Tank 
crew  training  can  thus  be  broken  down  into 
independent  and  specialized  phases. 

• Training  of  the  driver  who  receives 
orders  and  comments  from  the  tank  command 

• Training  of  the  gunner  whose  role 
generally  overlaps  with  that  of  the  tank 
conmander. 

The  function  of  the  loader  can  be  ne- 
glected at  this  stage;  it  is  a very  simple 
function  and  does  not  require  the  same  train- 
ing resources.  Simulators  have  been  developed 
for  driver  and  gunner  training.  Their  prin- 
ciples are  described  below. 

DRIVING  SIMULATOR 

The  driving  orders  given  to  the  driver 
result  from  the  tank  conmander' s analysis  of 
the  near  and  distant  situations.  For  this, 
the  conmander  must  be  given  a wide  field  of 
view. 

Driving  orders  are  simple  orders  which  define: 

• the  rallying  point 

• the  speed,  fast  or  slow 

• if  necessary,  the  path  to  be  followed. 

The  driver  carries  out  these  orders 
using  the  near  field  of  view.  Except  for 
giving  new  orders,  the  tank  conmander  takes 
little  part  in  the  driving  process.  The 


driver  is  responsible  for  carrying  out  all 
orders  received;  he  has  considerable  inde- 
pendence of  action.  This  is  important  for 
defining  the  student/instructor  relationship 
in  the  simulator. 

The  conditions  for  good  driving  are  as 
follows: 

• Good  practical  knowledge  of  the  vehi- 
cle. The  driver  must  use  the  vehicle  to  the 
best  effect  in  all  circumstances,  even  in 
the  event  of  malfunctions.  The  simulator  - 
and  this  is  an  inherent  feature  of  all 
training  simulators  - must  faithfully  repro- 
duce the  behavior  of  the  vehicle.  This 
implies  good  reproduction  of  driving  forces 
and  good  modeling.  The  rest  is  conventional. 

• Good  situation  appraisal.  Note  that 
situation  appraisal  is  obtained  almost  ex- 
clusively from  visual  sources;  obstacles, 
road  bends,  terrain  difficulties,  judgement 
of  distances  are  often  appreciated  from  vis- 
ible details.  Visual  representation  of  the 
terrain  must  therefore  be  of  high  quality. 

A closed  circuit  television  system  is  used, 
servo-coupled  to  the  driving  controls  and 
filming  a 1/300-scale  terrain  model  (figure 
1).  It  is  important  that  the  model  is  rich 
in  details.  The  image  display  must  allow 
driving  in  the  head-up  or  head-down  position. 

• Correct  translation  of  the  orders 
received  into  driving  commands  which  take 
the  visually  perceived  terrain  difficulties 
into  account.  The  effects  of  driving  action 
are  perceived  in  two  ways: 

a.  The  effects  of  driver  actions 
alone;  acceleration,  deceleration, 
steering.  These  effects  are  produced 
by  movement  of  the  image,  movement  of 
the  platform,  and  by  sound  effects. 

b.  The  effects  due  to  terrain  irreg- 
ularities; these  are  foreseeable 
effects  whose  origin  is  not  the  driver 
but  whose  amplitude  is  strongly  depen- 
dent on  driving  conditions.  It  is 
absolutely  essential  that  these  dis- 
turbing effects  be  correctly  simulated 
for  the  driver  to  obtain  the  same 
response  from  the  simulator  that  he 
would  obtain  from  the  vehicle.  He  can 
then  appreciate,  by  being  subjected  to 
them  himself,  the  strains  his  driving 
imposes  on  the  vehicle  - especially 
during  rough  driving  battle  conditions. 


93 


The  cabin  motion  system  must  have  good 
dynamic  characteristics  and  a large 
angular  excursion  in  the  pitch  axis. 

The  most  important  disturbing  effect  is 
the  elevator  effect  caused  by  pitch 
motion  and  due  to  the  driver's  station 
being  forward  of  the  center  of  rotation. 
This  effect  is  obtained  in  the  simulator 
by  placing  the  cabin  forward  of  the 
pitch  rotation  axis  of  the  platfon, 
(figure  2). 

Instructor's  role 


However,  it  is  possible  to  define  the 
main  functions  of  a gunner  and  the  char- 
acteristics which  a gunnery  training  simu- 
lator should  possess. 

The  most  important  functions  are: 

• Operating  knowledge  of  the  weapon 
control  system 

• Visual  situation  appraisal  and  iden- 
tification of  the  target  designated  by  the 
tank  comnander 


The  simulator  instructor  can  take  the 
part  of  the  tank  commander. 

• He  will  give  driving  orders.  The  sim- 
ulator instructor  is  given  the  same  informa- 
tion he  would  have  in  a real  tank  (figure  3): 

a.  The  same  scene  as  viewed  by  the 
driver;  repetition  of  the  trainee's  TV 
picture  at  the  instructor's  station. 

b.  Knowledge  of  the  terrain  which  would 
possibly  be  obtained  in  a real  tank  from 
a panoramic  observation  of  the  terrain; 

a 1/10-scale  map  of  the  terrain  model, 
on  which  the  tank's  position  is  contin- 
uously displayed,  allows  the  instructor 
to  give  general  driving  orders  ("Follow 
the  road  for  one  mile"),  or  particular 
orders,  which  could  result  in  reality 
from  a change  in  a battle  plan  ("Turn 
right,  off  the  road").  It  is  important 
that  the  instructor  has  the  information 
he  needs  to  be  able  to  give  driving 
orders  in  good  time.  The  terrain  map 
gives  him  this  advance  information. 

• He  will  comment  on  the  driving.  A 
repetition  of  the  image  seen  by  the  driver  and 
of  the  main  driving  instruments  (speedometer, 
rpm  counter,  etc.)  allow  the  instructor  to 
comment  on  the  trainee's  actions.  Using  a set 
of  dual  controls,  he  can  even  demonstrate 
correct  driving  technique  by  driving  the  simu- 
lator from  the  instructor's  station. 

Automatic  evaluation  . 

The  simulator  digital  computer  can  also 
be  used  to  provide  automatic  evaluation  of  the 
trainee's  performance.  This  type  of  evaluation 
must  be  based  on  a number  of  criteria  selected 
with  the  user's  approval.  The  available  com- 
puting power  is  sufficient  to  allow  appropri- 
ate weighting  of  each  evaluated  parameter. 

GUNNERY  TRAINING  SIMULATOR 

The  gunner's  functions  and  their  overlap 
with  the  tank  commander's  functions  vary  con- 
siderably depending  on  the  type  of  tank  and  on 
Army  procedures. 


• Aiming  at  mobile  targets 

• Shooting  and  observation  of  the  shot 
in  an  extremely  disturbing  environment 

• Correction  of  aim  and  further  shoot- 


Use  of  the  weapons  system 

The  gunner's  station  is  faithfully 
reproduced.  The  most  important  features 
concern: 

• The  sighting  head;  field  of  view, 
magnification,  various  reticles 

• The  aiming  controls;  play,  thresholds, 
efficiency. 

The  gunner's  station,  which  is  sub- 
jected to  the  shock  of  simulated  recoil, 
must  be  rigid.  Uncoupled  from  the  image 
generating  system,  it  can  be  replaced  by  the 
gunner's  station  for  another  type  of  tank, 
giving  the  simulator  considerable  versatil- 
ity. 

Visual  situation  appraisal 

Shooting  at  a distant  moving  target  is 
difficult.  The  simulator  must  provide  this 
type  of  training.  Identification  by  the 
gunner  of  the  target  designated  by  the  tank 
commander  requires  a high  quality  image. 

A combat  environment  can  be  created  by 
a "photo  quality"  image  and  by  a target 
moving  realistically  in  a realistic  land- 
scape. The  representation  of  an  aggressive 
target  trajectory  musf  be  able  to  induce 
apprehension  in  the  trainee  and  the  desire 
to  neutralize  as  quickly  as  possible  the 
enemy  who  is  taking  advantage  of  the  country- 
side (hollows,  thickets)  to  create  a danger- 
ous situation. 

Aiming  - Rangefindinq 

The  main  difficulty  in  this  phase  is 
due  to  target  mobility.  Movement  of  the 
target  over  the  terrain  - track  and  speed  - 
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must  be  modifiable  by  the  instructor  to  avoid 
trainee  familiarity  and  to  grade  the  exercise 
difficulties  to  the  trainee's  ability  to  over- 
come them. 

It  is  important  that  sighting  and  track- 
ing errors  are  quantified  in  the  simulator. 

Shooting  - Observation  of  results 

The  main  objective  of  the  simulator  in 
this  phase  Is  to  teach  the  trainee  to  disre- 
gard the  perturbations  accompanying  shooting: 

• recoil 

• noise 

• hot  air 

• smoke, 

and  to  observe  the  tracer  and  the  impact  to  be 
abl*.  to  correct  the  following  shot. 

The  classroom  simulator  is  superior  in 
its  ability  to  simulate  the  disturbing  envi- 
ronment which  is  one  of  the  important  diffi- 
culties of  this  phase  of  training.  The 
trainee  must  overcome  this  difficulty  before 
he  fires  his  first  real  shot. 

The  shell  trajectory  is  accurately  sim- 
ulated and  includes  the  following  effects: 

• environmental  effects;  external  tem- 
perature, wind,  etc. 

• weapon  effects;  dispersion,  barrel 
wear,  etc. 

Instructor's  role 

The  instructor  can  take  the  part  of  the 
tank  commander  in  three  essential  phases: 

• Target  designation.  As  the  instructor 
himself  selects  the  sequences  - type  of  tar- 
get, initial  position  in  range  and  bearing 
from  the  tank,  type  of  course  - he  can  then 
take  the  following  action  depending  on  the 
Army  procedure: 

Bring  the  weapon  to  bear  and  desig- 
nate the  target  or, 

give  the  gunner  the  orders  and  infor- 
mation allowing  him  to  bring  the  weapon 

to  bear,  identify  the  target,  and  shoot. 

• Tracking  and  shooting  at  a moving  tar- 
get. The  instructor  has  a repeated  display  of 
the  picture  seen  by  the  gunner;  the  landscape, 
the  target,  and  the  reticles.  He  is  therefore 
in  a position  to  comment  on  the  trainee's 
actions  at  any  time.  His  assessment  of  per- 


formance is  supported  by  a quantified  in- 
dication of  the  gunner's  errors  with  respect 
to  the  ideal  aim. 

• Post-shooting  commentary.  The  instruc- 
tor bases  his  comments  on  information 
obtained  from  two  sources: 

~ The  repeated  display  of  the  picture 

seen  by  the  gunner. 

Display  of  the  aiming  error  and 

impact  error  memorized  by  the  computer. 

The  gunner  therefore  receives  the  same 
directives  and  comments  from  the  instructor 
that  he  would  receive  in  reality  from  the 
tank  commander. 

Instructional  aids 

The  simulator  uses  a digital  computer. 
All  effects  (trajectory,  masks,  etc.)  are 
computed. 

Data  are  thus  available  to  give  the 
instructor  the  following  information: 

• numerical  readout  for  the  three  most 
important  parameters;  the  time  taken  by  the 
trainee  to  effect  an  operation,  aiming 
error,  and  impact  error. 

• exercise  evaluation  aids;  display  of 
the  ideal  aiming  axis,  freeze  of  the  shell 
at  the  point,  for  example,  where  it  passes 
through  the  vertical  plane  containing  the 
target. 

The  simulator  can  also  be  used  to 
provide  automatically  an  objective  assess- 
ment of  trainee  performance;  all  the  neces- 
sary parameters  exist  and  the  computing 
power  is  adequate. 

CONCLUSION 

The  simulator  designs  described  above 
are  fully  suited  to  efficient  classroom 
training  of  the  driver  and  gunner.  Each 
can  be  trained  in  his  own  function  and  get 
the  same  assistance  (orders  and  advice) 
from  the  instructor  as  he  would  from  his 
tank  commander. 

For  loader  training,  a very  simple 
special-purpose  trainer  would  be  quite  ade- 
quate. 

Training  of  the  tank  commander  in  his 
full  command  function  must  cover  not  only 
his  relation  with  the  other  crew  members 
but  also,  and  above  all,  the  tactical  situ- 
ations he  will  face.  Simulation  of  these 
situations  would  require  an  evolving,  com- 
plex display  including: 


i. 
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• a wide  angle  of  view  including  both 
near  and  far  fields 

• multiple  targets  so  that  the  risk  from 
each  can  be  assessed 

• friendly  vehicles  so  that  the  support 
potential  of  each  can  be  assessed. 


At  the  present  time,  it  is  considered 
that  classroom  simulators,  although  particu- 
larly cost-effective  for  training  drivers 
and  gunners,  are  less  cost-effective  for 
training  tank  commanders. 


Figure  2.  Cabin  motion  system 


Figure  1.  Visual  system  - Optical  probe 


Figure  3.  Instructor’s  station 
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INTRODUCTION 

An  experimental  investigation  is  being 
conducted  to  determine  the  feasibility  of 
using  scanned  lasers  to  generate  and  display 
real  world  scenes  for  military  training 
applications.  The  technical  objective  is  to 
provide  high  resolution  tactical  scenes  over 
a continuous  wide  field  of  view. 

The  US  Army  and  Air  Force  are. supporting 
the  development  of  a breadboard  laser 
camera  and  display  system  to  investigate  the 
capability  of  the  system  and  applicability 
of  this  approach  to  military  training  re- 
quirements. Completion  of  the  breadboard 
system  is  scheduled  for  third-quarter  1978. 

The  prime  contractor  is  American 
Airlines  Incorporated.  Redifon  Flight 
Simulation  Limited,  as  subcontractor,  is 
conducting  the  main  body  of  the  work,  and 
the  Sira  Institute,  England,  is  providing 
expertise  on  optical  and  electronic  systems. 

Image  Generation 

Before  considering  the  complete  scanned 
laser  visual  system,  the  basic  principle  of 
the  image  generation  system  will  be  described 
with  comparison  to  a conventional  television 
system.  The  basic  components  of  a convention- 
al television  system  are  illustrated  on  the 
left  side  of  Figure  1.  The  object  to  be 
televised  is  illuminated  and  the  reflected 
light,  over  a field  of  view  set  by  the  lens, 
is  picked  up  by  a television  camera  which 
generates  a video  signal  for  viewing  on  a 
picture  monitor.  The  signal  Is  generated  as 
the  electron  beam  in  the  television  pickup 
tube  scans  the  target  in  raster  format. 

On  the  right  side  of  Figure  1,  the 
same  object  is  scanned  by  what  may  be  called 
a laser  camera.  The  laser  camera  projects 
a laser  beam,  scanned  in  raster  format,  onto 
the  object.  The  video  signal  is  generated 
by  a photocell  placed  where  the  lamp  was 
placed  in  the  conventional  television  system. 

There  are  several  important  points  that 
should  be  noted  in  this  comparison.  Perhaps 
the  most  important  point  is  that  exactly  the 
same  image  appears  on  both  picture  monitors. 
This  is  true  as  long  as  the  generating  and 
reproducing  scanning  processes  are  synchro- 
nized and  the  field  scanned  by  the  laser  beam 


is  the  same  as  the  field  of  view  of  the 
television  camera. 

Second,  the  point  of  perspective  or 
viewpoint  with  the  laser  system  is  the  point 
from  which  the  scanned  laser  beam  leaves 
the  laser  camera  and  not  the  photocell. 
Movement  of  the  laser  camera  produces  normal 
perspective  views  just  as  if  it  were  a 
normal  television  camera. 

Third,  lighting  effects  can  be 
achieved  with  appropriate  placement  of 
photocells.  Just  as  in  a conventional  model 
board  visual  system,  where  multiple  lamps 
are  used  to  evenly  illuminate  a terrain 
model,  so  in  a scanned  laser  visual  system 
multiple  photocells  can  be  used  to  produce 
a scene  that  appears  to  be  evenly  illum- 
inated. A concentration  of  photocells  can 
be  used  to  generate  di  -■'ctional  lighting 
effects  and  fiber  optics  with  photocells 
behind  the  model  can  be  used  to  generate 
cultural  lights. 

Now  that  the  basic  principle  of  image 
generation  has  been  established,  the 
complete  scanned  laser  visual  system  will 
be  discussed. 

Laser  Visual  System  Description 

Figure  2 illustrates  the  complete 
scanned  laser  system.  As  in  a conventional 
camera  modelboard  visual  system,  the  gantry 
moves  the  laser  camera  in  x,  y(and  z 
coordinates  along  the  simulate’!  flight  path 
over  the  terrain  modelboard  Instead  of 
a lamp  bank,  there  is  a small  number  of 
photomultipliers  which  collect  the  laser 
light  reflected  from  the  model. 

The  laser  camera  uses  two  high  power 
ion  lasers,  an  argon  laser  to  provide 
green  and  blue  primary  colors,  and  a Krypton 
gas  to  provide  the  red  primary  color.  The 
output  beams  from  the  lasers  are  combined 
and  fed  through  flexible  fiber  optics  to 
the  laser  camera. 

The  laser  camera  contains  line  and 
frame  scanners  which  scan  the  beam  in  a 
raster  format  with  the  scanning  lines 
running  vertically.  In  the  preferred 
system  about  4400  vertical  scan  lines  are 
used  in  an  Interlaced  scan.  The  probe 
from  which  the  light  is  emitted  is  slender. 
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allowing  close  approach  to  the  terrain  model 
surface  and  between  vertical  features  on  the 
model . 

The  photomultipliers  are  arranged  in 
groups  of  three,  or  triads;  one  of  each 
triad  sensitive  to  red  light,  one  sensitive 
to  blue,  and  one  to  green.  All  the  red 
signals  are  sunned,  as  are  the  blue  and  green, 
to  produce  red.  blue  and  green  video  signals. 
The  video  signals  are  processed  with  blanking, 
gamma  correction,  aperture  correction, 
etc.,  and  fed  to  the  display. 

At  the  display,  the  three  primary  color 
beams  again  are  generated  by  two  lasers,  but 
before  being  combined,  the  beams  are  mod- 
ulated by  light  modulators  driven  by  the 
video  signals.  A flexible  fiber-optics 
light  guide  is  used  to  convey  the  composite 
modulated  beam  to  the  laser  projector. 

The  projector  uses  line  and  frame  scanners, 
accurately  synchronized  to  those  in  the 
laser  camera,  to  project  the  desired  scene. 

Since  it  is  necessary  to  project  the 
beam  onto  the  spherical  screen  without 
distortion,  direct  projection  cannot  be  used 
as  the  projection  point  cannot  be  coincident 
with  the  pilot's  head.  Accordingly,  as 
illustrated  in  Figure  3,  the  emerging  beam 
is  reflected  first  from  a spherical  mirror, 
giving  overall  correct  display  geometry  in 
the  wide-angle,  high-resolution  image. 

Field  of  View  and  Resolution 

The  video  bandwidth  in  the  system  can  be 
100  MHz  rather  than  5 MHz  for  a broadcast 
type  closed-circuit  television  system. 
Therefore,  approximately  twenty  times  as 
many  picture  elements  can  be  generated  and 
displayed  as  with  normal  television.  Various 
tradeoffs  are  possible  between  field  of  view 
and  resolution;  the  higher  the  field  of  view 
the  lower  the  resolution  and  vice  versa. 

Based  upon  current  military  training 
requirements  and  reasonable  development 
goals,  a 175  degree  by  60  degree  field  of 
view  was  selected  for  the  experimental 
breadboard  model.  With  this  field  of  view 
and  a 100  MHz  video  bandwidth,  a resol- 
ution of  5 arc  minutes  should  be  achieved. 

Optical  Layout 

In  order  to  cover  a 175  degree  field  of 
view  on  the  model  board,  the  frame  scan  is 
accomplished  by  rotating  the  frame  scan  prism 
from  which  the  laser  beam  emerges  about  an 
axis  normal  to  the  model  surface.  This  means 
that  the  field  angle  to  be  relayed  through 
the  scanning  probe  is  60  degrees  rather  than 
175  degrees,  and  fairly  conventional  optics 
can  be  used.  The  principle  of  the  scanning 
probe  is  illustrated  in  Figure  4. 


The  basic  principle  of  the  complete 
wide-angle  scanning  system  involves  passing 
the  laser  beam,  containing  red,  blue,  and 
green  components,  through  a line  scanner 
which  uses  a rotating  mirror  drum  to  deflect 
the  beam  in  a sawtooth  deflection  approx- 
imately 132,000  times  per  second.  The 
deflected  beam  is  then  passed  through  a 
derotation  prism  and  a wide-angle  lens 
which  expands  the  vertical  angle  to  60 
degrees.  Finally,  the  beam  is  deflected 
by  the  rotating  frame  scan  prism  to  emerge 
as  a raster  of  approximately  4400  vertical 
scan  lines.  The  derotation  prism  runs  at 
one-half  the  speed  of  the  final  prism,  to 
keep  the  scanned  line  vertical  as  the 
final  prism  rotates.  Using  a single  frame 
scan  prism  would  produce  a 360-degree 
frame  scan,  but  a nominal  180-degree  scan  is 
obtained  by  using  two  prisms.  To  allow  for 
a small  percentage  of  dead  time  during 
which  the  black  level  of  the  video  signals 
is  established,  the  actual  horizontal  field 
of  view  will  be  175  degrees. 

At  the  end  of  each  180-degree  rotation 
of  the  prisms,  the  line  scanned  beam  has  to 
be  switched  rapidly  by  a galvanometer 
mirror  between  these  two  prisms  so  that 
the  emerging  beam  is  always  in  the  forward 
direction.  This  process  of  changing  the 
beam  from  one  prism  to  the  next  inverts 
the  line  scan  so  that  the  beam  deflected 
by  the  galvanometer  has  to  be  taken  through 
a further  prism  for  correction. 

The  optical  layout  in  the  projector  is 
similar  to  that  of  the  probe  as  far  as  the 
line  and  frame  scanning  processes  are 
concerned.  As  mentioned  earlier,  the 
emerging  beam,  covering  175  degrees  by  60 
degrees,  is  directed  first  onto  a spherical 
mirror  and  then  onto  the  spherical  screen 
for  viewing  by  the  pilot.  The  display 
layout  Is  illustrated  in  Figure  3. 

Roll , Pitch,  Heading 

The  next  aspect  of  the  system  to  be 
considered  is  attitude  control.  In  a 
conventional  camera  model  visual  system, 
the  optical  probe  has  servo-controlled 
optical  elements  to  change  pitch  and  roll, 
and  the  complete  lower  part  of  the  probe 
structure  is  usually  rotated  about  an  axis 
normal  to  the  model  surface  to  provide 
heading  changes.  With  the  laser  probe, 
the  prisms  from  which  the  beam  emerges  are 
already  rotating  continuously.  By  altering 
the  phase  of  this  rotation  at  the  probe  in 
relation  to  the  corresponding  rotation  at 
the  display,  heading  changes  can  be  sim- 
ulated. 

In  order  to  allow  close  approach  to 
the  model  surface  and  easy  passage  between 
obstacles,  simulation  of  roll  and  pitch 
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Figure  3.  Wide-Angle  Laser  Display  System 


Figure  4.  Principle  of  Wide-Angle  Laser  Probe 


at  the  display  is  preferred.  There  is  plenty 
of  room  at  the  display  for  the  mechanisms 
necessary  to  achieve  this,  as  shown  in 
Figure  3. 

There  are  several  advantages  of 
simulating  roll  and  pitch  at  the  display. 
First,  objects  that  appear  within  the 
vertical  field  of  view  continue  to  be 
displayed  during  pitch  maneuvers  since  the 
entire  display  format  is  pitched;  Second, 
targets  are  not  lost  off  the  top  or  bottom 
of  the  screen  during  roll  maneuvers  since 
the  entire  display  format  is  rolled;  and 
tnird,  it  is  not  necessary  to  waste 
television  scanning  lines  in  reproducing 
blank  sky.  The  sky  can  be  simulated  with 
an  auxiliary  optical  projector  attached  to 
the  ma’n  projector. 

The  primary  disadvantage  of  simulating 
pitch  and  roll  at  the  display  is  that  an 
area  of  terrain  beneath  the  aircraft  cannot 
be  viewed  at  any  attitude  and  parts  of  this 
dead  area  appear  under  extreme  conditions 
of  roll  or  pitch. 

Focus 

The  next  aspect  of  the  proposed  scanned 
lase-  system  to  be  considered  is  that  of 
focussing  the  beam  on  the  terrain  model. 
Focussing  of  the  beam  on  the  display  screen 
is  constant  and  presents  no  problem. 

Depth  of  field  is  a very  important 
consideration  in  conventional  camera  model 
visual  systems.  The  optical  probe  attached 
to  a conventional  television  camera  has  an 
entrance  pupil  whose  size  determines  the 
depth  of  field  in  which  objects  will  appear 
to  be  in  sharp  focus.  The  larger  the 
aperture  the  worse  the  depth  of  field,  but 
the  smaller  the  aperture  the  more  the 
resolution  is  limited  by  diffraction. 

With  the  laser  system,  in  which  the 
scanned  laser  beam  leaves  an  exit  pupil  to 
strike  the  model,  the  same  basic  tradeoff 
exists.  The  laser  system  has  the  advantage, 
however,  that  the  laser  beam  can  be  put 
through  any  chosen  aperture.  This  is 
unlike  the  conventional  camera  model  system 
which  needs  a minimum  aperture  to  get 
enough  light  through  for  a noise-free 
picture.  Another  significant  advantage  of 
the  laser  system  is  that  only  one  spot  on 
the  model  is  illuminated  at  a time.  There- 
fore, changes  of  focus  and  aperture  can  be 
accomplished  during  scanning  to  give  optimum 
results.  By  contrast,  within  a conventional 
camera  model  system,  the  whole  field  of 
view  is  continuously  imaged  and  techniques 
to  improve  the  depth  of  field  must  operate  on 
the  whole  field  at  once. 


Two  main  types  of  focus  correction  may 
be  used  in  the  scanned  laser  visual  system-, 
the  choice  depends  on  the  visual  mission. 

For  missions  where  the  modelboard  may  be 
considered  to  be  flat  for  focussing  purposes, 
the  tilt  lens  techniques  used  for  conven- 
tional television  optical  probes  may  be 
adopted.  An  analysis  shows  that  good 
results  will  be  obtained  by  using  a high 
aperture  tilt  lens  in  the  optical  syster 
such  that  the  beam  is  focussed  at  a short 
distance  when  it  scans  the  near  part  of  the 
model  and  at  a greater  distance  when  it 
scans  the  far  parts.  The  absence  of  roll 
and  pitch  motions  in  the  probe  makes  the 
design  easier. 

For  missions  where  the  model  board  can 
not  be  considered  to  be  flat,  the  instanta- 
neous focus  may  be  changed  dynamically  to 
give  optimum  focus  for  each  object  struck 
by  the  scanning  beam. 

The  technique  for  carrying  this  out 
is  to  use  electrooptic  cells  to  vary  the 
width  and  convergence  of  the  laser  beam 
before  scanning.  At  the  exit  pupil  of  the 
laser  probe  the  emerging  beam  is  rapidly 
focussed  on  the  object  being  scanned  and 
the  width  of  the  beam  is  varied  to  change 
the  depth  of  focus. 

Figure  5 shows  the  laser  beam  passing 
through  an  electrooptic  cell.  The  control 
signal  is  applied  to  the  cell  which  can 
switch  the  polarization  of  the  emerging 
beam  through  90  degrees.  The  beam  is  passed 
through  a calcite  lens  which  has  different 
refractive  indices  for  the  two  polarizations. 
Therefore,  two  different  degrees  of  conver- 
gence of  the  beam  can  be  obtained. 

By  using  up  to  three  such  cells,  eight 
different  focus  conditions  are  sufficient 
to  give  smooth  change  of  focus  with 
distance.  By  correct  design  of  this 
focussing  system,  the  increased  convergence 
of  the  beam  needed  to  focus  on  near  objects 
can  be  linked  with  a decrease  in  pupil 
size  to  give  a better  overall  result. 

CONCLUSIONS 

An  advanced,  high-resolution  wide-angle 
visual  system  for  military  flight  simulation 
has  been  described.  The  basic  feasibility 
has  been  established  but  the  practical 
realization  of  the  system  remains  to  be 
proven.  Satisfactory  conclusion  of  this 
work  will  provide  a substantial  advance  in 
visual  simulation. 
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Three  stage  system 


Figure  5.  Electrooptic  Method  of  Focussing 
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INTRODUCTION 

In  the  last  ten  years,  we  have  witnessed 
a fortyfold  increase  in  the  edge  processing 
capability  of  real-time  digital  image  gener- 
ation (DIG)  systems.  While  it  would  appear 
unlikely  that  the  next  decade  will  produce 
another  fortyfold  increase  in  capacity,  one 
thing,  at  least,  seems  clear.  Like  some  in- 
satiable science  fiction  monster,  today's 
DIG  system  is  devouring  data  bases  at  a 
prodigious  rate  that  promises  to  be  ever- 
increasing.  To  satisfy  this  gargantuan 
demand,  as  well  as  the  desire  of  users  for 
more  detailed  and  realistic  simulation, 
greater  emphasis  will  have  to  be  placed  on 
the  data  base  generation  process.  Indeed, 
we  have  already  seen  the  beginnings  of  this 
trend  in  recent  procurement  specifications 
such  as  that  for  the  DIG  for  the  FB- 111 
simulator,  <D  which  not  only  specifies 
a total  data  base  storage  capacity  several 
times  the  size  of  the  deliverable  data 
base,  but  also  requires  the  delivery  of  a 
data  base  generation  facility,  thus  hinting 
at  the  Government's  intent  to  fill  up  this 
spare  storage  capacity  after  delivery.  In 
recognition  of  the  substantial  effort  re- 
quired to  generate  visual  simulation  data 
bases,  there  is  even  a quantitative  specifi- 
cation of  data  base  generation  speed. 

A comparable  situation  exists  in  the 
radar  simulation  field,  and  it  is  interesting 
to  observe  what  has  transpired  there.  The  need 
for  greater  modification  capability  and 
higher  resolution  has  led  to  the  adoption 
of  the  digital  radar  land  mass  simulation 
(DRLMS)  system  over  the  earlier  analog 
(transparency)  system  far  faster  than  has 
been  the  case  with  visual  simulation.  Data 
base  generation  for  DRLMS  started  with 
automated  scanning  and  digitization  of 
existing  analog  RLMS  transparencies,  and 
is  now  tending  to  standardize  on  data  com- 
piled by  the  Defense  Mapping  Agency 
Aerospace  Center  (DMAAC)  in  St.  Louis,  Mo. 

At  DhAAC,  there  has  been  a substantial  in- 
vestment in  equipment  and  personnel  to 
facilitate  the  encoding  of  photographic  and 
topographic  data,  primarily  for  radar  sim- 
ulation purposes. 


The  applicability  of  this  data  for  for- 
ward-looking infrared  (FLIR)  and  low-light- 
level  television  (LLLTV)  sensor  simulation 
has  apparently  been  recognized,  <2)  and  from 
there  it  is  a natural  step  to  extend  its  use 
to  visual  simulation.  Not  only  would  this 
reduce  the  cost  of  visual  data  base  gener- 
ation, but  it  would  also  result  in  correlated 
imagery  spanning  a wide  array  of  sensing 
equipment.  This  paper  describes  a system 
which  has  been  developed  to  utilize  DMAAC 
source  data  for  DIG  visual  simulation;  it 
also  describes  the  laboratory  real-time  DIG 
system  used  for  validation  of  the  results. 


LABORATORY  DIG  SYSTEM 

Concurrent  with  the  development  of  the 
data  base  generation  system  was  the  develop- 
ment of  a real-time  laboratory  DIG  system, 
which  served  as  the  test-bed  and  demonstration 
vehicle  for  the  data  base  generation  develop- 
ment. The  laboratory  DIG  hardware  will  be 
briefly  described  to  illuminate  the  environ- 
ment in  which  the  data  base  must  operate, 
although  continuing  development  has  since 
resulted  in  more  powerful  capabilities. 


Figure  1 is  a block  diagram  of  the  labo- 
ratory DIG  system.  Operation  of  the  system  is 
controlled  by  a Datacraft  (now  Harris)  6024/5 
general-purpose  computer.  This  same  computer 
was  the  central  controller  for  the  data  base 
generation  system.  The  computer  communicates 
directly  with  the  frame  calculator,  whose 
function  is  to  translate,  rotate,  clip,  and 
project  data  base  face  boundaries  onto  the 
computed  image  plane,  and  store  the  results 
in  the  double-buffered  edge  list.  The  frame 
calculator  operates  on  object  descriptions 
extracted  from  the  active  data  base  (ADB) 
store,  a 16K  x 48-bit  core  memory  loaded 
through  an  alternate  data  path  by  the 
general-purpose  computer. 

Edge  list  contents  for  an  entire  frame's 
computation  are  processed  on  a line-by-line 
basis  by  the  scanline  computer  to  produce 
a sequence  of  visible  scanline  segments, 
which  are  converted  to  video  information 
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Figure  1.  Laboratory  Dig  System 


by  the  video  generator.  This  is  displayed 
on  a high-resolution  color  monitor  viewed 
through  a collimating  mirror-beamsplitter  dis- 
play system. 

As  far  as  features  and  capabilities  are 
concerned,  the  rystem  handles  about  5,600  face 
boundaries  in  real-time  (30  frames  per  second) 
and  generates  1,000  scan  lines,  each  with 
1,000  resolution  elements.  Up  to  256  edge 
intersections  can  be  accommodated  per  scan 
line,  and  this  level  can  be  sustained  on  every 
scan  line.  Smooth-shading  is  incorporated  to 
provide  a rounded  appearance  to  curved  sur- 
faces. At  any  one  time,  up  to  64  distinct 
colors  may  be  displayed;  each  color  is 
specified  by  18  bits,  6 each  in  the  red, 
green,  and  blue  primaries,  to  form  a poten- 
tial color  space  of  more  than  262,000  possible 
hues.  Horizontal  and  vertical  edge  smooth- 
ing techniques  have  been  included  to  minimize 
quantization  effects. 

The  system  uses  a priority  list  technique 
for  resolving  visibility  contentions  between 
objects  which  overlap  when  projected  onto  the 
image  plane.  This  approach  constrains  objects 
to  be  convex,  so  as  to  preclude  the  overlap- 
ping of  faces  of  the  same  priority.  Up  to 
256  priorities  may  be  specified.  Object 
priorities  are  typically  assigned  in  real- 
time by  the  general-purpose  computer,  using  a 
separating  plane  approach.  (3>4)  Types  of 
objects  which  may  be  specified  include  two- 
dimensional  objects  (parallel  to  the  ground 
plane),  three-dimensional  objects  (solid  or 
oblique  to  the  ground  plane),  point  lights, 
strings  of  point  lights,  and  lines.  Lights 
can  be  specified  to  be  either  1 scanline  by 
1 resolution  element  or  2 scanlines  by  2 
resolution  elements  in  extent.  Lines  can  be 
either  1 or  2 elements  wide.  Objects  which 
are  not  lines  or  lights  may  either  have  their 
face  intensities  computed  dynamically,  as 


a function  of  sun  position  and  face  orienta- 
tion, or  be  specified  to  be  constant.  This 
latter  capability  has  been  used  to  minimize 
real-time  computation  requirements  and  to 
provide  for  unusual  sensor  characteristics, 
such  as  are  found  in  FLIR  or  LLLTV  simulation. 
Owing  to  a particular  choice  of  clipping 
algorithm,  all  non-light,  non-line  faces  are 
constrained  to  be  convex  (this  constraint  has 
been  removed  in  later  designs). 

OATA  BASE  GENERATION  SYSTEM  HARDWARE 

The  physical  emodiment  of  the  data  base 
generation  system  (DBGS)  is  depicted  in 
Figure  2.  The  central  element  is  the  central 
processing  unit  (CPU),  which  is  the  same  one 
used  for  the  real-time  DIG  system.  It  in- 
cludes 48K  24-bit  words  of  1 -microsecond  core 
memory.  The  DIG  and  display  are  also  included 
as  part  of  the  DBGS  hardware,  since  debug- 
ging and  final  proofing  are  an  integral  and 
necessary  part  of  the  data  base  generation 
process.  In  addition,  in  conjunction  with 
the  joystick  and  controls,  it  provides  a 
medium  for  interactive  data  base  modifica- 
tion, as  described  in  a later  section. 

Peripheral  devices  connected  to  the  CPU, 
aside  from  the  normal  teletypewriter,  card 
reader,  and  line  printer,  are  a 28-megabyte 
(9  million  words)  moving-head  disk,  used 
for  data  base  and  program  storage,  and  a 
9-track  magnetic  tape  system,  used  for 
backing  up  the  disk  and  providing  ad- 
ditional flexibility  to  interchange  data 
bases.  It  is  also  the  input  device  for  the 
DMAAC  data  base. 

The  display  terminal  is  a Tektronix  Model 
4010-1,  consisting  of  a storage  screen  dis- 
play and  alphanumeric  keyboard,  which  in- 
cludes hardware  vector  and  character  gener- 
ation. Graphic  input  is  also  permitted 
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Figure  2.  Data  Base  Generation  System  Hardware 


through  thumbwheel  control  of  a crosshair 
cursor.  This  device  serves  as  the  primary 
unit  for  graphic  interaction  and  for 
geometry  verification  prior  to  real-time 
display  with  the  DIG.  Connected  to  the  dis- 
play terminal  is  a Tektronix  Model  4610 
hard  copy  unit,  which  produces  facsimile 
copies  of  any  information  on  the  display 
screen.  This  is  used  to  provide  automatic 
documentation  of  the  data  base. 

The  digitizing  tablet  is  a Science  Acces- 
sories Corporation  Model  GP-2  Graf/Pen, 
with  12-bit  resolution  over  a 36-  by  36-inch 
active  area.  This  is  used  as  the  primary 
input  device  for  the  modification  of  the 
data  base,  and  for  the  entry  of  new  features 
or  geometry  not  contained  in  the  DMAAC  source 
data . 

THE  DMAAC  DATA  BASE 

The  DMAAC  digital  data  base  (DDB)  was 
conceived  to  satisfy  the  requirements  for 
digital  radar  land  mass  simulation  (DRLMS), 
initially  on  Project  1183.  (5,6)  At  the 
time  of  this  work,  the  DDB  consisted  of  two 
distinct  portions:  an  elevation  file,  which 

defined  predominant  ground  contours  by  means 
of  elevation  values  at  evenly  spaced  (in 
latitude  and  longitude)  grid  points,  and 
a cultural  file,  which  consisted  of  ortho- 
graphic outlines  of  cultural  features,  along 
with  some  natural  features  such  as  lakes  and 
rivers.  It  is  the  latter  file  on  which  the 
present  work  is  based,  there  having  already 
been  some  work  done  on  processing  grid 
point  information  to  produce  a DIG-compati- 
ble  data  base.  <7>8) 


The  cultural  file  is  organized  into  manu- 
scripts, rectangular  (in  latitude/longitude) 
areas  at  a given  data  resolution  level.  Of 
these  there  are  three,  the  finest  being  50- 
foot  resolution.  Within  each  manuscript, 
each  feature  is  assigned  a unique  feature 
analysis  code  (FAC)  number,  and  is  encoded 
geometrically  by  a directed  sequence  of 
vertices,  defined  in  geocentric  coordinates, 
and  a predominant  height,  which  is  equal  to 
zero  for  ground  level  (2-D)  features.  Ad- 
ditional descriptive  information  carried 
along  with  the  feature  includes  the  feature 
identification  (FID),  a number  which  deter- 
mines the  feature's  general  category  (tennis 
court,  commercial  building,  water  tower, 
etc.),  surface  material  type,  percent  of  roof 
cover,  and  percent  of  tree  cover. 


It  was  considered  important  that  the 
trial  DIG  data  base  consist  of  as  much  detail 
as  possible.  At  the  time  of  this  work,  only 
the  57,000-square-mile  Project  1183  data  base 
was  available;  the  most  detailed  manuscript 
therein  (Level  3B)  consisted  of  I square  mile 
of  downtown  Las  Vegas,  and  contained  about 
4,300  vertices.  A plot  of  the  manuscript 
is  shown  in  Figure  3.  The  decision  to  use 
this  manuscript  was  met  with  grudging  ac- 
ceptance by  the  data  base  modeler,  who 
reluctantly  volunteered  to  conduct  on-site 
data  gathering  activities  to  supplement  the 
DMAAC  DDB. 

DATA  BASE  GENERATION  SYSTEM 

The  data  base  generation  process,  as  well 
as  the  software  modules  required  to  implement 
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DATA  TAPE  REFORMMATING  PROGRAM 


Figure  3.  DMAAC  Level  3B  Plot  of  Down- 
town Las  Vegas 

the  process,  is  illustrated  in  Figure  4. 

The  elements  of  this  system  are  discussed 
below,  and  consist  of  the  Data  Tape  Reformat- 
ting Program,  the  Data  Base  Enhancement  Pro- 
gram, the  Data  Base  Compiler,  the  Perspective 
Line  Drawing  Routine,  the  Demo  Program, 
and  ancillary  programs. 


•The  DMAAC  DDB  was  furnished  on  a magnetic 
tape  prepared  on  a UNIVAC  1108,  a 36-bit 
machine.  The  function  of  the  Data  Tape  Re- 
formatting Program  (DTRP)  was  to  search  the 
tape  for  the  appropriate  manuscript,  con- 
vert all  items  for  compatibility  with  the 
24-bit  word  length  of  the  Datacraft  CPU, 
and  transform  the  geocentric  vertex  co- 
ordinates to  a rectilinear  system  in  a 
plane  tangent  to  the  earth  at  the  south- 
west corner  of  the  manuscript.  The  output 
of  DTRP  is  written  onto  a disk  file  (DTRP 
file)  for  convenient  access  by  the  Data 
Tape  Interpretation  Program  (DTIP).  DTRP 
is  executed  only  once  for  a given  manuscript, 
and  involves  no  human  interaction. 

DATA  TAPE  INTERPRETATION  PROGRAM 

The  DTRP  file  contains  a combination  of 
2-D  and  3-D  features,  of  which  the  3-D  fea- 
tures must  be  "extruded"  out  of  the  ground 
plane  to  the  extent  of  their  predominant 
height.  As  in  the  case  of  the  Level  3B  manu- 
script, feature  outlines  which  should  ideally 
be  smoothly  curved  (e.g.,  the  Landmark  Tower) 
are  ragged  and  uneven,  and  contain  unneces- 
sary vertices.  In  addition,  the  manu- 
script contains  many  more  vertices  than  the 


Figure  4.  Data  Base  Generation  Process 
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DIG  system  is  capable  of  handling:  of  the 
4,300  vertices  in  the  Level  3B  manuscript, 
approximately  2,400  are  contained  in  the 
145  3-D  features.  When  these  features  are 
extruded  from  the  ground,  an  additional 
9,600  face  boundaries  will  be  created, 
for  a total,  including  2-0,  of  13,900 
face  boundaries,  without  even  considering 
those  that  are  added  to  force  convexity. 
Consequently,  a considerable  number  of 
vertices  must  be  eliminated  in  order  to 
remain  within  the  DIG  processing  capacity, 
and  the  remaining  vertices  must  be  chosen 
for  efficient  and  accurate  portrayal  of 
feature  outlines,  as  well  as  for  subjec- 
tively important  qualities  such  as  symmetry 
and  regularity.  This  process  requires  a 
high  degree  of  human  involvement,  which  is 
facilitated  by  the  Data  Tape  Interpretation 
Program  (DTIP)  in  conjunction  with  the 
storage  screen  display  terminal. 

Using  DTIP,  the  data  base  modeler  works 
on  one  feature  at  a time,  first  selecting  it, 
either  by  referencing  its  FAC  number  or  by 
positioning  the  graphic  cursor  anywhere  with- 
in the  feature  boundary,  and  then  modifying 
its  geometry  as  required,  using  an  ortho- 
graphic plot  of  the  feature  or  the  entire 
manuscript.  This  modification  may  be  done 
by  deleting  or  moving  existing  vertices,  or 
by  adding  new  ones.  An  alternate  mode  allows 
complete  respecification  of  a feature  through 
the  sequential  placement  of  its  defining 
vertices.  In  all  cases,  vertices  are  refer- 
enced or  placed  by  positioning  the  graphic 
cursor  and  depressing  the  "space"  bar  on  the 
keyboard. 

On  the  5-  by  7-inch  display  terminal 
screen,  a 44-inch  square  area  on  the  left  is 
reserved  for  the  graphic  p’ots.  The  right- 
hand  scale  of  the  screen  is  reserved  for  the 
display  of  the  menu,  or  listing  of  allowable 
actions  at  each  step  of  the  modeling  process. 
Each  action  is  identified  with  a digit  (0-9) 
and  is  selected  by  activating  the  appropriate 
digit  key  on  the  keyboard.  Additional  dis- 
play fields  identify  the  name  of  the  file 
being  plotted  from,  and  the  scale  of  the  plot. 
The  use  of  the  former  field  arises  from  the 
identical  formats  of  the  DTF.P  and  DTIP 
(output)  files;  consequently,  either  file  may 
be  used  as  the  starting  point  for  modifying 
a feature  description.  The  need  for  the 
latter  field  is  due  to  the  capability  at 
various  steps  to  redefine  the  extent  of  the 
plot,  either  by  zooming  in  or  out  by  a factor 
of  two,  or  by  arbitrarily  defining  new  extents 
with  the  graphic  cursor. 

When  the  mooeler  is  satisfied  with  the 
appearance  of  a feature  (at  least  in  an 
orthographic  view),  he  may  declare  it  to  be 
"complete."  At  this  point,  the  feature 
boundary  is  not  necessarily  convex.  Since 


this  is  harmful  from  two  standpoints  (non- 
convex  faces  and  nonconvex  objects  are  not 
allowed),  steps  must  be  taken  to  insure 
orthographic  convexity.  This  is  done  auto- 
matically in  DTIP  by  adding  internal  edges 
connecting  selected  vertices,  so  that  the 
feature  is  divided  into  a set  of  convex 
faces.  The  result  is  plotted  on  the  display 
screen  to  allow  the  modeler  to  determine 
whether  the  "convexitization"  is  optimum 
(i.e.,  minimum  number  of  internal  edges); 
he  may  wish  to  delete  or  move  vertices  so 
that  fewer  faces  are  generated,  or  he  may 
wish  to  define  the  internal  edges  himself, 
a capability  which  has  been  provided  because 
the  present  convexitization  algorithm  is 
not  optimum. 

When  all  is  complete  and  satisfactory, 
the  modeler  then  "saves"  the  feature.  This 
passes  the  feature  description  to  the  Object 
Generation  Program,  after  first  verifying 
that  the  feature,  if  3-D,  does  not  overlap 
any  existing  3-D  features  in  the  DTIP  file. 
Were  this  to  be  so,  overlapping  objects 
would  be  generated;  these  are  not  allowable 
with  the  priority  list  technique. 


OBJECT  GENERATION  PROGRAM 

The  Object  Generation  Program  (0GP)  is 
actually  a subroutine  called  by  the  DTIP 
mainline,  and  after  execution  it  returns  to 
DTIP  to  allow  work  on  additional  features. 

The  function  of  0GP  is  to  convert  the  feature 
descriptions  from  the  DTRP/DTIP  format  (which 
takes  advantage  of  the  orthographic  nature  of 
the  vertex  infonnation  to  reduce  storage)  to 
a true  3-D  data  base  file  (DBF)  format,  after 
first  expanding  3-D  features  by  adding  "walls" 
to  the  "roofs." 


The  DBF  format  (see  Figure  5)  is  a hi- 
erarchical list  structure,  consisting  of  fea- 
ture, object,  face,  vertex  reference,  vertex 
coordinate,  and  vertex  normal  beads  (the  lat- 
ter only  for  smooth-shaded  objects);  each 
bead,  or  cell,  consists  of  a contiguous  (in 
memory)  sequence  of  words  which  define  the  re- 
quisite information  for  that  level  of  the 
structure,  as  well  as  pointers  to  the  next 
bead  at  the  same  level  and  the  first  bead  at 
the  next  lower  level.  This  type  of  format 
lends  itself  to  convenient  addition, 
deletion,  or  revision  of  information. 


In  addition  to  the  mechanical  task  of 
converting  file  formats,  0GP  also  calls  a 
series  of  routines  which  provide  automatic 
hardcopy  documentation  of  the  feature,  in- 
cluding objects,  faces,  and  vertices.  This 
documentation,  is  extremely  valuable  during 
later  debugging. 
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DATA  BASE  ENHANCEMENT  PROGRAM 

Not  all  significant  features  can  be  ex- 
tracted from  the  DMAAC  DDB,  and  some  of  those 
that  can  are  not  well  represented  by  simply 
expanding  them  in  three  dimensions.  For 
example,  the  Las  Vegas  Convention  Center  dome 
is  extremely  recognizable  in  the  real  world, 
but  not  in  the  DMAAC  data  (Figure  6);  the 
Landmark  Tower,  while  it  is  represented  as  a 
separate  feature  in  the  DDB  (Figure  7), 
would  not,  when  mechanically  expanded,  allow 
the  accurate  portrayal  of  the  actual  struc- 
ture. In  cases  like  this,  source  material 
other  than  the  DMAAC  DDB  must  be  used.  The 
Data  Base  Enhancement  Program  (DBEP)  allows 
the  entry  of  this  information  into  the  data 
base,  and  is  also  the  principal  tool  for  the 
modification  of  the  data  base. 

DBEP  is  controlled  exclusively  from  the 
digitizing  tablet,  using  predefined  areas  of 
the  digitizing  surface  as  "buttons"  which 


Figure  6.  DMAAC  Plot  of  Las  Vegas 
Convention  Center 


the  data  base  modeler  may  activate  at  the 
appropriate  times.  In  addition,  the  tablet 
is  also  used  as  a pure  graphic  input  device, 
allowing  entry  of  specially  formatted  draw- 
ings. For  2-0  features,  these  drawings 
depict  a plan  view  of  the  faces  to  be  enter- 
ed. For  3-D  features,  both  plan  and  side 
views  are  represented  on  the  drawing,  one 
above  the  other,  as  shown  in  Figure  8.  To 
digitize  a face  from  this  drawing,  corre- 
sponding pairs  of  vertices  from  the  two  views 
are  entered  in  counterclockwise  order  around 
the  face.  A perspective  view  of  the  solid 
figure  obtained  from  the  drawing  in  Figure  6 
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is  shown  in  Figure  9.  While  less  elegant 
than  some  other  approaches  for  3-D  data 
entry,  <9>  this  scheme  was  nonetheless  easy 
to  program  and  proved  quite  workable  in 
practice. 

Aside  from  the  ability  to  enter  new  fea- 
tures, DBEP  also  provides  feature  modification 
and  manipulation  capabilities.  For  example, 
existing  faces,  objects,  or  features  may  be 
translated,  rotated,  or  both;  face  colors  may 
be  selected;  vertex  normal  vectors  may  be 
automatically  computed  as  an  average  of  face 
normals,  or  may  be  specified  numerically; 
duplicate  copies  or  mirror  images  of  existing 
features  or  objects  may  be  created;  and 
existing  features,  objects,  faces,  or 
vertices  may  be  deleted  from  the  data  base. 
Extensive  error  checking  is  conducted  at  all 
steps  to  insure  object  and  face  convexity  and 
face  planarity,  among  other  conditions.  At 
any  time  partial  documentation  may  be  re- 
quested for  an  intermediate  check  on  data 
validity,  and  automatic  documentation  is  gen- 
erated upon  saving  a feature  in  the  DBF.  The 
documentation  routines  used  are  the  same  as 
those  called  by  OGP,  and  also  provide  a check 
for  linear  separability  of  objects  within  the 
feature.  In  all,  DBEP  provides  a powerful 
tool  for  data  base  generation  and  modifica- 
tion. 

DATA  BASE  COMPILER 

Prior  to  being  loaded  into  the  DIG  ADB 
core,  the  data  base  must  be  converted  from  the 
easily  updatable  list  format  in  the  DBF  to 
the  highly  compressed  format  required  by  the 
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Figure  8.  Three  Dimensional  Data  Entry  Figure  9.  Perspective  View  of  3-D  Solid 
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DIG  processing  hardware,  a function  performed 
by  the  Data  Base  Comil er  (DBC).  In  the  pro- 
cess, default  face  colors  and  intensities 
must  be  assigned  for  those  faces  extracted 
directly  from  the  DMAAC  DOB,  which  contains 
no  color  information.  At  present,  this  as- 
signment is  based  only  on  FID  number;  where 
the  modeler  wishes  this  assignment  to  be 
overridden,  he  must  manually  enter  a color/ 
intensity  combination  with  DBEP.  Intensities 
for  other  sensors  (FLIR,  LLLTV ) can  also  be 
derived  during  this  stage  of  the  processing. 

In  addition  to  the  data  conversion,  the 
DBC  performs  two  other  important  functions: 

2-D  priority  consolidation,  and  separating 
plane  generation.  Since  2-D  faces  are  all 
in  the  ground  plane,  they  can  all  be  as- 
signed the  same  priority  number,  provided 
they  do  not  overlap,  without  any  deleterious 
effects  appearing  in  the  display.  However, 
some  2-D  faces  purposely  overlap  others  to 
minimize  the  number  of  face  boundaries  re- 
quired. 2-D  consolidation  seeks  to  con- 
serve priorities  by  collecting  2-D  faces 
into  the  minimum  number  of  priority  classes, 
where  all  members  of  a class  are  assigned 
the  same  priority  number.  In  the  Las  Vegas 
data  base,  35  2-D  features  were  reduced  to 
only  four  priority  classes,  thus  allowing 
a greater  number  of  3-D  objects. 

Whereas  the  priorities  of  2-D  objects  can 
be  assigned  during  the  data  base  compilation, 
those  of  3-D  objects  depend  on  their  position 
and  arrangement  relative  to  the  observer. 
Consequently,  computation  of  these  must  be 
deferred  to  real-time.  The  concept  of 
separating  planes  allows  the  convenient  pre- 
calculation of  object  priorities  using  the 
modest  facilities  of  the  general -purpose 


computer.  The  computation  of  the  separating 
planes,  and  the  tree  structure  which  defines 
their  relationships  to  each  other  and  to  the 
3-D  objects,  is  performed  off-line  by  the 
DBC  using  a refined  trial-and-error  technique. 

To  allow  for  experimentation  with  dif- 
ferent versions  of  features  or  objects,  the 
DBC  accepts  as  input  a card  deck  that  defines 
which  features  and/or  objects  to  omit  from 
the  current  compilation.  In  addition,  the 
time  of  day  must  be  specified  to  allow  the 
computation  of  face  intensities  based  on  face 
orientation  and  calculateo  sun  position. 

PERSPECTIVE  LINE-DRAWING  ROUTINE 

Final  proofing  of  data  base  geometry, 
prior  to  actual  display  with  the  DIG,  is  done 
through  the  generation  of  perspective  line 
drawings,  with  hidden-line  removal  based  on 
the  separating  plane/priority  list  approach. 

In  order  to  allow  convenient  user  specifi- 
cation of  the  viewpoint  position  and  orienta- 
tion for  generating  the  line  drawing,  the  Per- 
spective Line-Drawing  Routine  first  creates 
a hard-copy  orthographic  map  of  the  data 
base  on  the  storage  screen  display,  using 
just  the  objects  obtained  from  the  last 
previous  data  base  compilation.  The  user 
then  tapes  this  map  to  the  digitizing  tablet, 
locates  the  map  by  digitizing  two  corners, 
and  then  digitizes  the  positions  of  both  the 
viewpoint  and  the  point  being  viewed,  first 
in  X-Y  directly  on  the  map,  and  then  in  2 
using  a scale  along  the  side  of  the  map. 

The  routine  then  computes  object  priorities 
using  the  separating  planes  previously 
compiled,  and  displays  the  objects  on  the  4010 
screen  in  priority  order.  A sample  of  the 
output  is  contained  in  Figure  10. 


Figure  10.  Perspective  Line  Drawing  of  Las  Vegas  Data  Base 


110 


DEMO  PROGRAM 

The  proof  of  the  process  is  actually 
viewing  the  data  base  in  real  time  on  the 
color  display,  under  the  control  of  the  Demo 
Program.  The  Demo  Program  performs  three 
primary  functions: 

1)  Aircraft  emulation  - using  joy- 
stick and  switch  inputs,  observer 
position  and  attitude  are  calculated 
according  to  simplified  aerodynamics; 

2)  Priority  calculation  - priorities  of 
3-D  objects  are  computed  by  travers- 
ing the  separating  plane  tree; 

3)  DIG  communication  - commands  and 
data  are  transferred  to  the  DIG  pro- 
cessing hardware  using  standard 
communication  subroutines. 

Aside  from  the  viewing  capabilities  pro- 
vided by  the  above  functions,  the  Demo  Pro- 
gram also  provides  a powerful  data  base 
debug  and  modification  capability.  By 
enabling  the  Color  Modification  portion  of 
the  program  with  a switch  at  the  control 
console,  a user  can  cause  a small  square 
cursor  to  be  generated  in  the  picture.  This 
cursor  is  responsive  to  joystick  motion,  and 
can  be  positioned  over  any  face  or  object. 

When  a button  on  the  joystick  handle  is  de- 
pressed, the  numbers  of  the  feature,  object, 
and  face  overlaid  by  the  cursor  are  printed 
on  the  teletypewriter.  Then,  by  means  of 
slew  switches  on  the  control  console,  the 
user  may  dynamically  change  the  color  and/or 
intensity  of  the  object/face  while  observing 
the  effects  on  the  display.  Future  enhance- 
ments of  this  capability  may  include  the 
ability  to  adjust  vertex  normals  for  smooth- 
shaded  objects. 


ANCILLARY  PROGRAMS 

iwo  ancillary,  but  important,  programs 
have  not  yet  been  discussed:  the  Button  Table 
Generator  and  the  Color  Map  Generator.  The 
Button  Table  Generator  allows  the  definition, 
using  the  digitizing  tablet,  of  the  coordi- 
nates of  buttons  used  by  DBEP.  In  addition, 
it  allows  the  checking  and  modification  of 
button  locations,  and  the  addition  of  new 
buttons.  The  output  of  the  Button  Table 
Generator  is  stored  on  a disk  file  accessible 
to  DBEP. 

The  Color  Map  Generator  allows  the  speci- 
fication of  the  64  possible  colors  which  com- 
prise the  permissible  color  space  at  any  one 
time.  This  specification  takes  the  form  of  a 
six-bit  value  for  each  of  the  red,  green,  and 
blue  (R,  G,  B)  components  of  the  desired 
color.  The  specifications  for  all  64  colors 
are  contained  in  a random-access  semiconductor 


memory  in  the  video  generator,  called  the 
color  table  or  color  map.  The  color  map 
can  be  thought  of  as  a catalog  of  paint  chips, 
from  which  the  data  base  modeler  can  select 
the  colors  of  faces  and  objects.  Each  chip 
has  a number,  from  0 to  63,  which  is  the 
address  of  the  color  map  entry  for  that  chip; 
this  number  is  what  is  carried  along  in  the 
data  base  to  represent  color,  and  during  the 
video  generation  process,  a table  lookup  is 
used  to  transform  from  this  number  to  the 
actual  CRT  drive  voltages.  The  color  table 
is  loaded  during  the  initialization  phase  of 
the  Demo  Program  from  the  color  map  in  the 
Color  Map  Disk  File,  which  is  generated  in 
turn  by  the  Color  Map  Generator. 

It  had  initially  been  thought  that  the 
color  map  entries  would  be  empirically  deter- 
mined using  the  actual  display.  However, 
a superior,  purely  analytical  approach  was 
developed  which  allowed  the  determination 
of  color  map  entries  directly  from  the  hue, 
value,  and  chroma  information  supplied  with 
each  color  chip  in  the  Munsell  Book  of 
Color,  do)  which  was  used  as  the  basis  for 
color  selection.  This  determination  took 
account  both  of  the  color  CRT  phosphor  capa- 
bilities and  the  transfer  function  of  the 
D/A  converters  and  gamma  correction  cir- 
cuitry in  the  video  generator.  The  results 
achieved  with  this  approach  were  outstanding, 
and  no  empirical  correction  was  required  to 
match  the  original  Munsell  colors  within 
human  discernibility  limits. 

RESULTS 

As  indicated,  the  DBGS  was  used  on  the 
DMAAC  Level  3B  manuscript  of  downtown  Las 
Vegas.  Using  DTIP,  the  original  4,300 
vertices  were  reduced  to  1,783,  the  result  of 
eliminating  about  60  features,  retaining  50 
in  their  original  form,  and  using  DTIP  to 
"outline"  another  50.  The  orthographic  data 
base,  after  all  work  with  DTIP,  and  includ- 
ing edges  added  for  convexity,  is  shown  in 
Figure  11 . 

After  this  step,  drawings  were  prepared 
and  digitized  for  another  13  features,  in- 
cluding the  Convention  Center  dome.  Circus 
Hotel,  Landmark  Tower,  and  Hilton  Hotel.  The 
resulting  3-D  data  base,  consisting  of  about 
5,300  face  boundaries,  is  shown  in  Figure  12, 
which  is  reproduced  from  a photograph  taken 
from  the  DIG  monitor.  Additional  photos  of 
this  data  base  are  contained  in  Reference  11. 

The  versatility  of  the  DBGS,  particularly 
DBEP,  is  illustrated  in  Figures  13-16,  which 
are  DIG  photographs  of  data  bases  for  the 
space  shuttle  orbiter,  a terrain-following 
range,  and  day  and  night  versions  of  the  San 
Jose  Municipal  Airport.  These  data  bases 
started  with  drawings,  so  DTRP  and  DTIP  wete 
not  used. 


Ill 


FUTURE  DEVELOPMENT 


Figure  11.  Las  Vegas  After  Modification 
with  DTIP 


Figure  12.  Las  Vegas  Data  Base 


Figure  13.  Space  Shuttle  Orbiter 


Many  shortcuts  were  taken  in  the  design 
of  DTRP  and  DTIP  to  take  advantage  of  known 
characteristics  of  the  Level  3B  manuscript, 
including  geographic  location  and  maximum 
number  of  vertices.  The  design  of  these 
modules  should  be  generalized  to  allow  han- 
dling of  arbitary  manuscripts.  This  will  in- 
clude matching  of  features  from  adjoining 
manuscripts,  and  from  manuscripts  at  differ- 
ent levels  of  description.  Some  automation 
can  be  incorporated  into  the  program  to,  for 
example,  eliminate  redundant  collinear  (or 
nearly  so)  vertices  based  on  a user-supplied 
tolerance.  In  addition,  data  may  have  to  be 
organized  differently  to  satisfy  the  require- 
ments of  dynamic  data  base  retrieval  programs, 
which  will  probably  be  inevitable  in  any  non- 
trivial simulation  application.  These  changes 
will  of  course  propagate  through  the  other 
programs  of  the  DBGS,  principally  the  DBC. 


CONCLUSION 

The  data  base  generation  system  described 
herein  demonstrates  the  feasibility  of  semi- 
automated  methods  for  the  production  of 
cultural  DIG  data  bases  for  visual  simulation. 
This  approach  depends  on  a pre-existing 
source  of  digitized  data  covering  a large 
geographic  area,  and  is  made  economical 
through  increased  automation  of  decisions  and 
inclusion  of  visually  significant  information 
(e.g.,  colors,  intensities,  textures)  in  the 
source  data.  While  human  intervention  in  the 
data  base  generation  process  will  never  be 
totally  eliminated,  it  can  be  reduced  to 
those  areas  in  which  human  facilities  are 
superior  to  those  of  machines:  pattern  recog- 
nition, perceptual  judgement,  and  determina- 
tion of  importance  to  training. 


Figure  14.  Rolling/Mountainous  Terrain 
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Figure  15.  San  Jose  Municipal  Airport 
(Day  Version) 


Figure  16.  San  Jose  Municipal  Airport 
(Night  Version) 
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A MEW  VISUAL  SIMULATION  TECHNIQUE  FOR  PILOT  TRAINING 


CARL  J.  VORST 

McDonnell  Douglas  Electronics  Company 


INTRODUCTION 

Visual  simulation  has  become  a ma.ior  factor 
in  replacing  aircraft  hours  with  simulator 
hours  for  pilot  training.  Rapid  increase  in 
visual  use,  made  possible  by  computer  gener- 
ated imagery  (CGI)  technology,  is  in  turn  a 
major  factor  in  increased  simulator  use. 
Generally  replacing  television  model  and  film 
approaches,  CGI  has  demonstrated  '^aining 
flexibility  and  low  cost  when  pro;  rly  ap- 
plied. This  paper  presents  application  of  a 
new  CGI  technology  particularly  suited  to  the 
problem  of  pilot  training;  that  technology 
being  the  heart  of  the  VITAL  system  developed 
by  McDonnell  Douglas  Electronics  Company. 

A general  CGI  characteristic  is  that  equip- 
ment complexity  is  directly  related  to  in- 
stantaneous scene  content.  Of  the  two  gen- 
eral CGI  categories,  raster  and  calligraphic 
(stroke  written)  scan,  both  require  similar 
size  data  base  storage  and  processing  capa- 
bility. An  important  difference  is  that 
raster  scan  requires  a significant  amount  of 
parallel  processing  while  calligraphic  pro- 
cessing is  serial.  This  translates  to  the 
potential  for  considerable  hardware  savings 
with  calligraphic  use. 

Consideration  of  visual  requirements  demon- 
strates that  overall  instantaneous  scene 
complexity  is  roughly  constant  in  a pilot 
training  application.  Figure  1 is  an  example 
which  illustrates  this  by  comparing  two  stens 
in  a landing  sequence.  At  six  miles  from 
touchdown  (Figure  la)  a large  number  of  light 
points  are  in  view  but  very  few  solid  sur- 
faces may  be  seen.  As  the  approach  contin- 
ues, light  points  pass  out  of  view,  while 
visible  surfaces  grow  in  size  and  number. 

At  runway  threshold  (Figure  lb),  only  a few 
light  points  are  in  view  while  numerous  sur- 
faces of  significant  size  are  present. 


Figure  la.  6 Miles  from  Threshold 


Figure  lb.  P. . W.  Threshold 


The  sequential  nature  of  calligraphic  tech- 
niques fits  hand-in-hand  with  such  scene  re- 
quirements in  that  writing  time  may  split  be- 
tween precision  point  and  surface  display  as 
required  by  instantaneous  scene  content. 

While  calligraphic  light  point  display  has 
for  some  time  been  successfully  performed 
with  a jump-settle-unblank  sequence,  effi- 
cient generation  of  smoothly  textured  sur- 
faces has  been  the  subject  of  much  research. 


VITAL  surface  generation  (Figure  2)  imple- 
ments a calligraphic  technique  uniquely  de- 
veloped for  the  VITAL  application,  which 
eliminates  earlier  problems.  The  surfacing 
device,  descriptively  named  "Raster  Blaster," 
is  pipelined  between  a general-purpose  com- 
puter and  a high-speed  color  graphics  display 
unit.  The  Blaster  is  designed  to  generate 
precision  points  and  surfaces  of  uniform  tex- 
ture with  minimum  computer  intervention.  It 
contains  the  complete  functions  beam  steer- 
ing, blanking,  surface  shading,  along  with 
the  control  of  color,  intensity,  and  focus. 

A detailed  description  of  the  device  and  its 
design  follows. 
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Figure  2.  Vital  Shape  Generation 


VITAL  System  Concepts 

The  Virtual  Image  Takeoff  and  Landing  (VITAL) 
system  was  formulated  as  a solution  to  a 
particular  visual  simulation  problem.  That 
problem  was  one  of  twilight  and  night  take- 
off and  landing  under  a variety  of  visibility 
conditions.  Several  unique  design  concepts, 
particularly  in  the  area  of  surface  genera- 
tion electronics,  were  used  to  satisfy  the 
requirements  posed  by  this  application. 

The  system  (Figure  3)  is  partitioned  into 
four  elements:  1)  system  inputs,  2)  image 

generation  equipment,  3)  picture  controller 
(Raster  Blaster),  and  4)  display  electronics. 
This  third  element,  the  Raster  Blaster,  is 
the  key  to  system  performance.  Its  function 
is  twofold;  first,  to  generate  control  sig- 
nals for  display  of  points,  and  second,  to 
qenerate  control  signals  for  display  of  sur- 
faces. Because  of  it'  special  advantages 
the  surface  generation  concept  will  be  dis- 
cussed in  detai  1 . 

A surface  is  defined  in  terms  of  its  outline, 
represented  by  a segmented  right-edge  vector 
and  a segmented  left-edge  vector.  The  beam 
is  moved  in  alternate  directions  at  a con- 
stant velocity  and  with  a fixed  separation 
between  horizontal  lines  within  the  surface 
bounds.  Constant  beam  velocity  and  separa- 
tion are  critical  to  achieving  uniform  inten- 
sity over  the  entire  enclosed  area.  Effi- 
cient use  of  the  deflection  system  is  made  by 
writing  in  both  directions  rather  than  using 
a blanked  retrace.  To  aid  in  maintaining 
constant  beam  velocity  and  line  separation, 
the  beam  is  blanked  outside  the  shape  bounds 
and  time  allowed  for  beam  settling. 

In  the  Blaster,  surface  reconstruction  is 
performed  by  the  X and  Y computers,  and  by 
the  sweep  generator.  The  X computer  incre- 
mentally generates  a representation  of  the 
left  and  right  edges  of  a surface  by  adding 
a AX  to  the  current  X value  each  time  tne 
beam  is  incremented  by  ohe  raster  line,  AY. 
Storage  is  provided  for  the  several  iX's  re- 
quired to  generate  a segmented  edge.  The  Y 
computer  generates-  current  vertical  beam 
position  and  compares  it  with  the  Y coordi- 
nates of  edge  vector  breakpoints.  The  break- 
point comparison  information  is  used  to 
select  the  proper  AX's  for  use  by  the  X com- 
puter. The  sweep  generator  defines  instan- 
taneous horizontal  beam  position.  Sweep 
direction  is  controlled  by  comparing  beam 
position  to  right  and  left-edge  position. 

To  reduce  processing  time  during  beam 
turnaround,  the  X and  Y computers  operate  as 
separate  pipelines,  calculating  data  ahead  of 
the  time  of  its  required  use  in  beam  position- 
ing. The  X computer  serves  a dual  purpose  in 
that  it  uses  portions  of  the  edge  logic  as  a 
1 ightpoint  pipel ine. 


Much  of  the  geometric  appearance  of  CGI 
scenes  is  removed  by  applying  shading  to  sur- 
faces. Figure  4 illustrates  the  effect  con- 
tinuous shading  has  on  scene  quality.  With 
shading,  a realistic  tapered  horizon  glow  is 
provided,  fading  smoothly  as  angle  above  the 
horizon  increases.  Intensity  taper  on  the 
runway  surface  provides  a depth  cue  and  may 
be  used  to  represent  the  effect  of  landing 
lights  on  surface  visibility. 

Intensity  taper  is  under  full  control  of 
software.  The  Z computer  uses  software  sup- 
plied shading  coefficients  in  combination 
with  horizontal  and  vertical  deflection  sig- 
nals to  produce  an  interpolation  of  intensity 
simultaneously  in  X and  Y.  To  represent  a 
background  illumination  in  conjunction  with 
intensity  taper,  as  would  be  found  in  a twi- 
light landing  scene,  the  Z computer  has  pro- 
visions to  clamp  intensity  to  a program  con- 
trolled minimum  level. 

Focus  control  logic  operates  on  the  CRT  focus 
amplifier  to  produce  light  points  of  a con- 
trolled size  regardless  of  color.  It  may 
also  be  used  for  cormanding  defocus  of  points 
or  shapes  as  required  by  special  applications. 
It  is  possible  to  expand  focus  control  to  in- 
clude a focus  taper,  much  like  the  intensity 
taper. 

Design  Considerations 

While  simple  in  concept,  numerous  considera- 
tions entered  into  the  final  design.  Design 
characteristics  of  special  importance  were 
first  defined.  These  characteristics  were 
matched  to  potential  implementations  and  then 
evaluated  through  methodical  testing.  Quite 
often  this  meant  scrapping  a design  and 
starting  over. 

Design  goals  of  special  importance  to  the 
VITAL  form  of  point  and  surface  generation 
were: 

for  points  - 

• provide  point  capability  for 
standard  approach  and  runway 
patterns  along  with  miscel- 
laneous background  lights  and 
landmarks. 

• allow  settling  time  that  is  a 
function  of  deflection  distance. 

• arrange  points  in  strings  to 
reduce  average  deflection  dis- 
tance. 

• pipeline  data  transfers  to 
minimize  effects  of  I/O  data 
rates. 

• provide  separate  adjustments 
for  color  geometry  (gain,  off- 
set, rotation,  brightness). 
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Figure  3.  The  Vital  System 
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b.  Tapered  Horizon  and  Runway  Intensity 


c.  Variation  of  Intensity  Taper  - Right  Outboard 
Landing  Light  On 

Figure  4.  Effects  of  Intensity  Taper 


for  surfaces  - 

• achieve  writing  rate  sufficient 
to  present  a horizon  glow,  run- 
way surface  with  standard  mark- 
ings, and  other  special  ele- 
ments such  as  taxiways  and  key 
bui ldi ngs. 

• maintain  intensity  uniformity 
over  a single  surface. 

• generate  a tapered  surface  in- 
tensity (surface  shading). 

• provide  good  edge  sharpness. 

• avoid  edge  ripple  and  face 
flicker. 

• maintain  a solid  appearance. 

• correct  for  differences  in 
transport  delays  between  de- 
flection and  intensity  chan- 
nels. 

For  the  most  part,  design  goals  for  point 
presentation  were  met  without  the  need  for 
fanfare.  Surface  generation,  however,  pro- 
vided some  interesting  design  problems.  A 
few  of  these  are  summarized  in  the  following 
paragraphs. 

Decreasing  line  density  increases  effective 
surface  writing  rate.  As  line  density  in- 
creases, beam  defocus  is  needed  to  eliminate 
visibility  of  individual  lines.  Defocus 
must  be  used  with  caution,  as  surface  edges 
tend  to  lose  definition.  Experiments  showed 
that  when  this  was  applied  to  small  surfaces, 
they  had  a definitely  defocused  appearance 
which  in  some  cases  was  actually  more  annoy- 
ing than  the  visibility  of  individual  lines. 
Reducing  line  density  also  increases  the 
detrimental  effects  of  edge  ripple  and  face 
flicker  on  small  surfaces.  This  approach  can 
still  be  used  effectively  if  applied  with 
care  to  drawing  only  large  surfaces  such  as 
horizon,  cloud  top  and  runway;  using  high- 
line  density  for  small  surfaces. 

Writing  on  retrace,  while  decreasing  surface 
display  time,  requires  precise  control  of 
beam  position  to  prevent  noise  effects.  Much 
of  the  design  effort  has  gone  towards  achiev- 
ing this  control.  As  can  be  seen  from  Figure 
b,  the  beam  must  be  moved  quickly  and  smooth- 
ly from  one  raster  line  to  the  next.  At  high 
deflection  rates  CRT  brightness  is  highly  de- 
pendent upon  beam  dwell  time.  Any  vertical 
overshoot,  when  positioning  to  a new  raster 
line,  generates  a potential  for  the  beam 
overwriting  an  area  covered  by  a previous 
raster  line.  Overwriting  alters  dwell  time, 
subsequently  altering  brightness,  much  as 
increasing  film  exposure  produces  a brighter 


photo.  The  regular  pattern  generated  is  very 
noticeable.  Vertical  undershoot  has  the  ef- 
fect of  reducing  uniformity  of  line  density. 
The  partial  overwriting  that  occurs  at  shape 
edges,  produces  edge  brightening  and  increases 
the  visibility  of  individual  line  pairs.  Care 
must  be  taken  to  select  a D/A  converter  and 
amplifier  combination  for  Y deflection  that 
minimizes  conversion  glitches,  so  as  not  to 
aggravate  the  settling  problems. 

Horizontal  deflection  linearity  is  equally 
critical  to  picture  quality.  Variations  in 
horizontal  writing  rate  produce  effective 
changes  in  dwell  time,  with  resulting  inten- 
sity variations.  In  addition,  nonlinear 
deflection  induces  errors  in  edge  placement. 
Horizontal  deflection  anomalies  result  from 
amplifier  response  factors  plus  some  charac- 
teristics that  are  inherent  to  the  sweep 
generation  method.  Amplifier  considerations 
are  similar  to  those  for  vertical  deflection. 
The  sweep  generation  method  must  be  free 
from  turnaround  instability  and  rate  varia- 
tions. Several  possible  methods  of  analog 
sweep  generation  are  illustrated  in  Figure  6. 
Digital  ramp  approaches  were  avoided  because 
of  the  high  update  rates  required  for  low- 
position  granularity. 

The  most  common  approach,  an  operational  am- 
plifier integrator  (Figure  6a),  while  pos- 
sessing excellent  linearity,  suffers  from 
output  spikes  on  turnaround,  resulting  in 
deflection  ringing.  Causes  are  twofold; 
input  to  output  coupling  through  the  feed- 
back capacitor  and  amplifier  small-signal 
bandwidth,  both  difficult  to  deal  with  at 
the  frequencies  of  interest. 

Alternately  charging  a capacitor  from  bal- 
anced voltage  sources  (Figure  6b),  while 
free  from  turnaround  spikes,  suffers  from 
rate  variations.  Reasonable  charging  lin- 
earity can  be  obtained  only  when  the  source 
voltage  is  much  larger  than  the  maximum 
capacitor  voltage. 

Alternately  charging  a capacitor  from  bal- 
anced current  sources  (Figure  6c),  offers 
both  good  linearity  and  freedom  from  turn- 
around spikes.  Practical  current  sources 
may  be  built  operating  with  voltages  very 
near  the  desired  peak  capacitor  charging 
voltages.  This  was  chosen  as  the  most  suit- 
able method  for  VITAL. 

Dwell  time  effects  on  brightness  and  edge 
precision  weighed  heavily  in  the  selection 
of  the  VITAL  writing  techniques  from  among 
those  shown  in  Figure  7.  Some  pros  and  cons 
of  techniques  considered  are: 

• Vector  outline  - Good  edge  defini- 
tion, but  no  texture. 


* 


c.  Write  on  Retrace-V  Overdamped 


NONUNIFORM  LINE  DENSITY 
CAUSING  BRIGHT  AREA 


d.  Write  on  Retrace-Y  Properly  Damped 


Figure  5.  Effects  of  Nonuniform  Line  Density 


b.  Capacitor  Charged  from  Voltage  Source  through  Resistance 


Figure  6.  Analog  Sweep  Generation 
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a.  Vector  Outline 


b-  Transformed  Vectors 


Truncated  Parallel  Vectors 
(End  Points  Bounded  by  Shape) 


e.  Truncated  Parallel  Vectors 
(End  Points  Outside  Shape 
Intensification  Bounded  by  Shape) 


Figure  7.  Some  Variations  of  Calligraphic  Surface  Generation  Techniques 
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• Transformed  vectors  - Transformation 
of  individual  vectors  is  relatively 
easy.  Volume  of  computations  is 
high.  Intensity  and  precision  con- 
trol is  poor  because  of  nonuniform 
line  density. 

• Modulated  Periodic  Waveforms  - Better 
texture  and  precision  uniformity 
than  transformed  vectors.  Suffers 
from  variable  dwell  time  resulting 

in  brightening  of  shape  edges.  Line 
density  varies  50  percent  between 
surface  edge  and  center. 

• Truncated  Parallel  Vectors  (bounded 
by  surface  edge)  - Capable  of  very 
uniform  texture  and  brightness  ex- 
cept at  edge  where  practical  deflec- 
tion bandwidth  limitations  restrict 
beam  position  control. 

• Truncated  Parallel  Vectors  (end 
points  outside  surface  edges,  in- 
tensification bounded  by  edges)  - 
Capable  of  very  uniform  texture 
and  brightness  over  entire  sur- 
face. Requires  precise  control 
of  blanking. 

The  latter  approach  was  taken  because  of  ex- 
treme importance  placed  on  surface  uniformity 
Any  regular  distortion  can  act  as  a strong 
distraction,  drawing  attention  from  the  scene 
itself.  Blanking  control  requires  considera- 
tions that  are  much  more  important  than  for  a 
raster  type  system.  Relative  timing  between 
the  blanking  and  horizontal  deflection  sig- 
nals is  critical.  With  a raster  scan  surface, 
any  timing  mismatch  results  in  positional 
offset  of  the  entire  surface  (Figure  8a). 

The  effect  of  timing  mismatch  on  a write-on 
retrace  surface  is  to  create  a shadow  effect 
on  the  edges  (Figure  8b).  The  VITAL  blanking 
signal  is  delayed  to  compensate  for  deflec- 
tion transport  delay,  thus  removing  the  edge 
shaoow.  Blanking  delay  is  adjustable  to  com- 
pensate for  individual  deflection  variations. 


-H^l— 


Understanding  the  nature  of  deflection  trans- 
port delay  is  necessary  for  proper  delay 
compensation.  The  horizontal  deflection 
chain  gives  the  dynamic  effect  of  a multi- 
plicity of  low-pass  filters.  Figure  9 gives 
the  output  of  a simple  low-pass  filter  with 
a ramp  input.  The  output  follows  the  input 
by  one  time  constant  except  near  the  origin, 
where  there  is  distortion.  The  blanking  de- 
lay compensates  for  the  combined  time  con- 
stants. Distortion  near  the  origin  produces 
a variable  delay  which  could  result  in  dif- 
ferent matching  requirements  for  small  and 
large  shapes.  Deflection  overscan  is  used  to 
linearize  the  output  before  unblanking,  mini- 
mizing delay  variations.  A tradeoff  must  be 
made  between  amplifier  bandwidth  (fj)  and 
overscan  time,  reduction  of  overscan  requir- 
ing an  increase  of  bandwidth.  VITAL  deflec- 
tion bandwidth  is  sufficiently  high  to  make 
linearity  compensation  delay  a secondary  fac- 
tor. 


Figure  9.  Distortion  of  Ramp  Input  by  a 
Simole  Low-Pass  Filter 
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CONCLUSIONS 

The  Raster  Blaster  has  demonstrated  a distinct 
set  of  advantages.  The  outstanding  advantage 
is  the  efficiency  of  serial  lightpoint  and 
surface  generation.  Serial  generation  means 
that  the  same  set  of  common  electronics  may  be 
used  to  generate  each  lightpoint  and  surface 
which  appears  in  the  scene. 

The  method  of  surface  generation  offers  sev- 
eral advantages  in  itself.  The  compressed 
description  of  a surface  by  equations  repre- 
senting its  outline,  means  low  data  rates  be- 
tween the  computer  and  Blaster,  freeing  the 
computer  to  perform  necessary  processing.  In 
addition,  the  effective  split  in  computation 
load  permits  more  efficient  use  of  the  com- 
puter's capabilities.  The  computer  performs 
the  relatively  complex  job  of  surface  outline 
transformation  and  windowing,  while  the 
Blaster  performs  the  simple  but  time  consum- 
ing operation  of  incrementally  generating  the 
surface  outline.  Thus,  for  a given  amount  of 
computation  power  applied,  a more  complex 
scene  may  be  produced. 


Uniform  surface  texture  gives  the  appearance 
of  solidity.  Combined  with  computer  con- 
trolled intensity  taper,  defined  differently 
for  each  surface,  illusion  of  depth  is  en- 
hanced, producing  an  unexpected  degree  of 
rea  1 i sm . 

The  Blaster  has  not  yet  realized  its  full 
speed  potential.  As  needed,  effective  sys- 
tem speed  may  be  increased  through  improve- 
ment of  the  computer  and  display  electronics. 
Computer  improvements  include  expansion  and 
the  addition  of  special-purpose  processing 
electronics.  Display  speed  improvements  can 
be  made  by  upgrading  the  deflection  system. 

What  was  designed  as  a solution  to  the  spe- 
cific problem  of  takeoff  and  landing  simula- 
tion has  rapidly  found  application  in  a di- 
verse and  growing  set  of  pilot  training  areas. 
A few  application  scenes  are  pictured  in 
Figure  10.  Ongoing  research  indicates  that 
this  new  and  yet  open-ended  technology  is  due 
to  produce  some  pleasant  surprises  in  the 
near  future. 
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b.  Carrier  Landing  Scene,  JFK  e.  Helicopter  Staging  Field  with  Tower  and 

Building  in  Foreground  and  Helicopter 
Landing  Pinnacle  in  Background 


c.  Multiple  Runway  Helicopter  Staging  Area  f.  Daylight  Scene  with  Random  Field  Pattern 

with  Maltese  Cross  Touchdown  Zone  Markers 

Figure  10.  Vital  Applications 
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PERSPECTIVE  ERROR  IN  VISUAL  DISPLAYS 


ROBERT  ENTWISTLE  and  NEIL  MOHON 
Naval  Training  Equipment  Center 


Perspective  in  photographic  reproductions 
is  one  of  the  most  important  visual  cues  to  a 
human  observer  primarily  because  it  is  almost 
always  used  subconsciously  to  make  some  deter- 
mination about  the  photographed  subject.  Many 
photographs,  especially  in  the  advertising 
industry,  introduce  significant  errors  in  the 
perspective  of  the  object;  but  that  error  is 
rarely  detected  by  the  conscious  mind  of  the 
observer. 

Consider  the  example  photographs  of  an 
automobile  shown  in  figure  1.  the  two  pic- 
tures were  made  with  the  automobile  in  exactly 
the  same  location  relative  to  the  background 
scene.  However,  quite  a different  impression 
is  relayed  to  the  mind  from  figure  la  than 
figure  lb.  Figure  la  was  made  using  a 200  mm 
focal  length  lens  on  a 35  mm  camera,  while 
figure  lb  was  made  using  a 35  mm  focal  length 
lens  on  the  same  camera.  Advertising  photo- 
graphs are  often  made  using  short  focal  length 
lenses  in  order  to  make  the  car  appear  longer, 
more  attractive,  or  to  emphasize  some  particu- 
lar detail . 

One  basic  anomaly  in  our  "advertising 
oriented”  mind  is  that  we  accept  the  erroneous 
perspective  of  figure  lb  without  question. 

Maybe  we  expect  it.  The  next  time  you  see  an 
automobile  advertisement,  or  any  advertisement, 
consciously  look  for  the  errors  in  perspective. 

The  problem  in  perspective  arises  because 
we  do  not  view  the  photographic  records  from  a 
proper  distance,  or  viewpoint.  Figure  la  will 
yield  correct  perspective  when  viewed  from  60 
cm;  and  figure  lb  will  be  correct  when  viewed 
from  10  cm. 

When  we  consider  the  environment  of  a 
military  training  simulator  in  conjunction 
with  its  ultimate  purpose,  it  becomes  impera- 
tive that  we  give  careful  attention  to  per- 
spective and  the  impression  it  makes.  In  a 
simulator  the  trainee  is  frequently  called 
upon  to  make  rapid  judgments  about  range, 
relative  size,  or  rate  of  movement  of  a target 
based  solely  upon  a photographic  type  display. 
Obviously  the  scene  should  have  as  little  error 
in  perspective  as  possible  in  order  for  the 
trainee  to  make  accurate  estimates.  Hence, 
perspective  is  a factor  in  the  training  value 
of  a display. 

We  have  designed  two  photographs  to  illus- 
trate this  concept  and  they  are  shown  as  figure 
2.  The  two  people  are  located  in  exactly  the 
same  relative  positions  in  both  photographs; 


and  yet  there  is  a dramatic  difference  in  the 
apparent  perspective  presented  to  a viewer. 
Figure  2a  was  made  using  a 200  inm  focal 
length  lens  on  a 35  mm  camera;  and  figure  2b 
was  taken  using  a 50  mm  lens  on  the  same  cam- 
era. If  two  situations,  similar  to  figure  2, 
were  presented  to  a novice  trainee  in  a simu- 
lator, it  is  quite  possible  that  he  would 
make  erroneous  judgments  in  range  and/or  rela- 
tive positioning.  If  a target  were  moving 
from  position  A to  position  B,  the  judgment 
of  the  trainee  as  to  velocity  and  direction 
of  movement  could  easily  be  distorted  for  the 
tv/o  situations. 

We  observe  from  figures  1 and  2 that  per- 
spective is  the  angular  relationship  between 
objects  in  the  scene  as  viewed  from  the  camera 
position.  When  a picture  is  made  from  a rel- 
atively short  distance  from  the  objects,  the 
angular  relationship  between  the  objects  is 
greater  than  if  the  picture  were  made  from  a 
longer  distance.  The  perspective  in  a picture 
is  exactly  correct  when  the  relative  angles  in 
the  photographic  reproduction  are  identical  to 
the  camera  viewpoint  angles  that  existed  in 
the  original  recording  geometry. 

The  correct  viewpoint  distance,  and  hence 
the  correct  viewing  angle,  may  be  derived  from 
the  optical  relationship  inherent  in  the  cam- 
era-projector-observer  system.  Figure  3 rep- 
resents the  entire  system  from  the  object,  0, 
through  the  camera,  through  the  projection 
(or  enlarger)  apparatus,  and  to  the  observer 
at  the  viewpoint,  V.  The  angle  subtended  by 
the  object  is  given  by 

tan  a = h/s , ( 1 ) 

where  h is  half  the  object  height  and  s is  the 
object  to  camera  lens  separation.  To  maintain 
this  angle,  a,  at  the  viewpoint,  we  require 
that 

h"/d  = tan  a.  (2) 

Equating  1 and  2 and  solving  for  d,  the  cor- 
rect perspective  viewpoint,  we  find 

d * h"s/h.  (3) 

From  the  magnification  relationship  of  the 
projector,  we  know 
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Using  this  in  equation  3,  we  find  that 

d = mh'  s/h.  (5) 

From  the  magnification  relationship  of  the 

camera , we  know  that 

h'/h  = -s'/s,  (6) 

or  that 

s'  = -h's/h.  (7) 

Now  substitute  equation  7 into  5 to  yield 
d = -ms ' . (8) 

The  lens  imaging  equation  states  that 
1/s  + 1/s'  = 1/f,  (9) 

or  rearranging 


We  have  used  the  concept  of  perspective 
error  ratio  to  analyze  a simulator  in  which 
riflemen  are  taught  marksmanship  against  a 
series  of  targets  approaching  them  on  a wide 
front.  The  requirement  involved  ten  trainees 
firing  onto  a wide-angle  screen  for  accuracy 
score.  Figure  4 illustrates  the  configura- 
tion of  the  simulator.  One  constraint  re- 
quired that  the  trainees  be  positioned  along 
a straight  line  25  meters  long  and  be  able  to 
fire  completely  across  their  50  meter  front. 
Because  we  had  to  have  satisfactory  screen 
brightness,  good  resolution,  and  a minimum 
screen  size,  we  wanted  to  move  the  screen  as 
close  as  possible.  The  result  was  a cylin- 
drical screen  of  30  meter  radius  having  view- 
ing positions  generally  indicated  by  the  three 
dots  0m,  10  m,  20  m,  to  represent  respective- 
ly positions  on-axis,  10  meters  off-axis,  and 
20  meters  off-axis.  From  the  0M  on-axis  view- 
ing position,  the  screen  subtends  30°  on 
each  side  of  the  optical  axis. 


s'  = f/(l  - f/s).  (10) 

Now,  substitute  equation  10  into  8 to  find 
d = -mf / ( 1 - f/s).  (11) 


But,  in  cameras,  it  is  almost  always  true 
that  s»>f;  therefore,  we  say  that 


(12) 


where  we  have  absorbed  the  negative  sign  into 
the  magnification  factor,  m.  Equation  12 
states  that  the  viewpoint  distance,  d,  of  a 
photographic  recording  for  correct  perspective 
is  found  by  multiplying  the  focal  length,  f, 
of  the  recording  camera  lens  by  the  magnifica- 
tion, m,  of  the  projector  (or  enlarger)  sys- 
tem. Note  that  this  viewpoint  must  be  on  the 
optical  axis  as  shown  in  figure  3 in  order  to 
maintain  the  correct  angular  relationships. 


Usually  a photograph  or  projected  image 
is  not  or  cannot  be  viewed  from  the  correct 
perspective  distance  (d).  The  actual  distance 
from  which  it  is  viewed  we  will  call  distance 
(D),  and  we  will  define  a new  parameter.  Per- 
spective Error  Ratio,  as 


PER  = d/D 


(13) 


Or,  substituting  equation  13  into  12, 
PER  = mf/D 


(14) 


Where  the  actual  distance  viewed  (D)  is 
larger  than  the  correct  perspective  distance 
(d)  we  obtain  a number  larger  than  one,  and 
where  it  is  smaller  we  obtain  a number  smaller 
than  one.  Multiplying  PER  by  100  we  may  ob- 
tain a value  for  the  percent  perspective  error 
(PPE)  which  may  be  useful  in  some  descriptions. 


Figure  5 illustrates  the  PER  across  the 
60°  screen  for  each  of  the  three  positions  of 
figure  4.  The  perspective  error  ratio  for 
position  0m,  which  is  directly  under  the  pro- 
jector lens,  is  one  everywhere  on  the  screen; 
i ,e. , this  is  the  position  defined  by  equa- 
tion 12  above.  The  PER,  from  equation  14, 
varies  considerably  across  the  screen  for  the 
other  two  positions  and  is  not  linear.  The 
errors  in  perspective  are  greatest  when  the 
trainees  are  firing  across  the  screen  to  the 
opposite  corner  of  the  simulator.  As  of  this 
writing,  there  are  no  definitive  results  as 
to  the  tolerable  limits  of  PER  in  a training 
simulator  --  or  any  photographic  type  scene. 

So  our  approach  is  to  minimize  perspective 
error  as  much  as  possible  by  pushing  PER 
toward  one  while  fulfilling  the  system  re- 
quirements. We  know  of  no  published  informa- 
tion on  the  effects  of  perspective  error  on  a 
trainee's  learning  and/or  accuracy.  To  date, 
the  best  approach  has  been  to  make  some  as- 
sumptions from  acceptable  perspective  error 
in  portrait  photography  and  apply  them  to 
this  simulator;  all  of  which  may  or  may  not 
be  valid. 

Perspective  is  a significant  visual  cue 
in  the  display  of  a training  simulator  because 
it  is  needed  to  make  accurate  estimates  of 
range,  size,  and  velocity  of  objects  in  the 
viewed  scene,  and  is  normally  used  in  a sub- 
conscious manner.  We  have  introduced  the 
mathematical  concept  of  perspective  error  ratio 
as  an  approach  to  measuring  and  evaluating  the 
perspective  in  a scene.  And  we  have  intro- 
duced a graphical  type  analysis  of  perspective 
error  ratio  to  easily  study  the  perspective 
error  in  a display.  We  propose  that  the  next 
step  is  to  produce  standard  objects  for  meas- 
uring experimentally  the  perspective  error  in 
a display  and  determining  what  tolerable  limits 
can  be  placed  upon  such  errors  for  various 
type  simulators. 
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Figure  2a.  Perspective  of  200  mm  focal  length 
lens  on  a 35  mm  camera. 


Figure  2b.  Perspective  of  50  mm  focal  length 
lens  on  a 35  mm  camera. 
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Figure  3.  Viewpoint  for  correct  perspective. 


Figure  4.  Simulator  configuration  with  three 
representati ve  viewing  positions* 


Perspective  Error  Ratio 


1.5 


Figure  5.  Perspective  Error  Ratio  (PER)  as  a 
function  of  scene  angle  for  three 
viewing  positions. 
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COMPENSATING  FOR  FLIGHT  SIMULATOR  CGI  SYSTEM  DELAYS 

G.  L.  RICARD,  D.  A.  NORMAN,  and  S.  C.  COLLYER 
Naval  Training  Equipment  Center 


INTRODUCTION 

Modern  flight  training  simulators  are 
usually  equipped  with  digital  computers  that 
measure  the  pilot's  control  activities,  deter- 
mine the  simulated  aircraft's  responses,  and 
provide  student  performance  data  for  the  in- 
structor. Recent  developments  have  made  it 
possible  to  extend  digital  processing  tech- 
niques to  the  generation  of  visual  images, 
allowing  an  entirely  computer-controlled 
training  environment  where  a flight  dynamics 
processor  calculates  the  responses  of  a simu- 
lated aircraft  and  a computer  generated 
imagery  (CGI)  system  presents  the  changing 
visual  scene  to  the  pilot  via  a graphics  dis- 
play. 

Digital  control  systems  can  encounter 
timing  problems  when  their  processing  involves 
numerous  calculations.  In  the  past,  the  time 
taken  by  a digital  machine  to  compute  the  dy- 
namics of  a flight  simulation  has  not  been  a 
serious  constraint  on  the  use  of  simulators. 
With  the  addition  of  CGI  systems,  however, 
the  delays  imposed  by  calculation  times  are 
having  an  effect  on  simulator  utilization. 

The  temporal  sequence  of  information  flow 
needed  to  update  a computer  generated  visual 
scene  is  depicted  in  Figure  1.  Usually  two 


measure  the  operator's  control  inputs  at  rates 
of  15  to  32  samples  per  second,  allowing  air- 
craft position  information  to  be  available 
with  minimal  delays  ranging  from  66  to  about 
31  milliseconds.  Currently  available  CGI 
systems  take  about  100  milliseconds  to  update 
their  images,  creating  total  system  delays 
from  approximately  100  to  200  milliseconds 
between  the  measurement  of  the  pilot's  inputs 
to  the  simulator  and  the  visual  presentation 
to  him  of  its  response.  Actual  delay  values 
may  vary  slightly  depending  upon  when  the  dy- 
namics processor  makes  the  results  of  its 
calculations  available  to  the  CGI  system. 

The  problem  introduced  by  adding  a CGI 
system  delay  to  delays  already  created  by  the 
dynamics  processor  of  a simulation  system  was 
highlighted  in  Device  2F90,  a TA4J  Operational 
Flight  Trainer,  when  it  was  noticed  (after 
the  addition  of  a CGI  visual  system)  that 
flyers  of  the  trainer  tended  to  produce  pilot 
induced  oscillations  on  the  last  leg  of  the 
carrier  approach  task.  In  that  device,  the 
severity  of  these  roll-axis  oscillations  was 
reduced  by  changing  the  aileron  response  to 
control  stick  deflection,  but  oscillations 
were  still  encountered,  especially  if  a large 
roll  input  was  made  during  the  last  1/8  mile 
of  the  carrier  approach  (O'Conner,  Shinn, 


PILOT  CONTROL 


Figure  1.  Sequence  of  Calculations  in  a Typical  CGI  Display  System. 


processors  are  used  ; one  to  control  the  simu- 
lator and  one  to  do  the  graphics  processing. 
The  simulator  controller  measures  the  pilot's 
control  inputs,  calculates  the  new  aircraft 
position,  and  makes  the  new  position  available 
to  the  qraphics  processor.  The  CGI  system 
then  obtains  the  new  position  of  the  simulated 
aircraft  and  calculates  a new  visual  image  to 
represent  it.  Present  simulation  systems  can 


and  Bunker,  1973;  or  Harris,  1975).  A similar 
situation  was  encountered  using  the  Advanced 
Simulator  for  Undergraduate  Pilot  Training  at 
Williams  Air  Force  Base.  Test  pilots  experi- 
enced an  undesirable  roll-axis  instability 
during  the  landing  approach  and  formation  fly- 
ing tasks  (Larson  and  Terry,  1975).  Presum- 
ably this  is  due  to  that  device's  126  to  193 
millisecond  total  transport  delay. 
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Recent  Work 

The  effects  of  inserting  various  types 
of  delays  into  feedback  control  systems  that 
contain  human  controllers  have  been  studied 
for  some  time,  and  it  is  well  recognized  that 
the  presence  of  delays  affects  the  controlla- 
bility of  systems.  Air  Force  sponsored  stud- 
ies on  first-and  second-order  lags  and  on 
transport  (dead-time)  delays  indicate  that 
the  control  of  compensatory  systems  that  have 
random  forcinq  functions  is  affected  by  any 
type  of  delay,  but  that  transport  delays, 
which  allow  no  system  response  for  the  dura- 
tion of  the  delay,  are  most  disruptive.  Be- 
cause pilot  performance  is  affected  by  short 
delays,  the  reduction  of  the  effects  of  trans- 
port delays  in  the  visual  systems  of  flight 
simulators  is  an  important  cons ideration. 


Research  at  the  National  Aeronautics 
and  Space  Administration  Lanqley  Research 
Center  has  assessed  the  effects  that  small 
transport  delays  have  on  pilot  performance 
when  those  delays  were  inserted  into  the 
visual  display  of  a flight  simulator.  Queijo 
and  Riley  (1975)  had  experienced  controllers 
perform  a "following"  task  in  a fixed-base 
simulator  where  a "target"  aircraft  underwent 
slow  sinusoidal  oscillations  of  altitude  and 
the  task  was  to  follow  the  target.  Dead-time 
delays  ranging  from  50  to  300  milliseconds 
were  inserted  before  the  visual  display.  The 
amount  of  delay  that  could  be  tolerated  was 
found  to  be  related  to  ratings  of  the  flying 
qualities  of  a variety  of  simulated  aircraft. 
Only  small  delays  could  be  allowed  in  high- 
performance  aircraft  simulations  before  their 
presence  was  objectionable  and  flying  perform- 
ance suffered.  When  the  subjects  flew  simu- 
lations of  aircraft  with  better  handling 
Qualities,  lonaer  delays  had  less  of  an  ad- 
verse effect  on  performance. 


These  results  were  extended  by  Miller 
and  Riley  (1976)  who  activated  the  motion 
base  of  the  simulator  and  varied  the  fre- 
quency of  the  altitude  changes  that  their 
pilots  had  to  follow.  As  might  be  expected, 
they  found  that  when  the  frequency  of  the 
target  plane's  oscillations  was  raised,  the 
subjects'  performance  under  even  small  delays 
became  poor.  They  also  found  that  the  use 
of  the  notion  base  extended  the  range  of  de- 
lays that  could  be  easily  accommodated.  The 
demonstration  that  either  the  handling  quali- 
ties of  the  simulated  aircraft  or  the  diffi- 
culty of  the  flying  task  can  determine  the 
extent  to  which  dead-time  delays  can  disrupt 
performance  indicates  the  need  to  develop 
compensations  for  CGI  visual  systems.  As 
we  try  to  simulate  more  sophisticated  air- 
craft or  to  teach  the  more  complicated  flying 
tasks,  we  might  expect  display  delays  to  be 
increasingly  detrimental. 


At  the  Naval  Training  Equipment  Center, 
Cooper,  Harris,  and  Sharkey  (1975)  have  mea- 
sured the  differences  in  piloting  performance 
that  display  delays  produce.  Using  the 
TRADEC  F-4  simulator  and  a carrier  landing 
task,  they  measured  the  amplitude  spectra  of 
the  forces  and  deflections  that  experienced 
pilots  exerted  on  the  simulator's  controls 
in  the  presence  and  absence  of  a 100  milli- 
second visual  system  delay.  By  subtracting 
the  spectra  obtained  under  no-delay  conditions 
from  similar  spectra  obtained  under  the  delay, 
they  obtained  a difference  spectrum  for  each 
control  axis.  This  spectrum  presents  the 
differences  (across  frequency)  of  the  pilot's 
control  inputs  that  characterize  control  in 
the  presence  of  a 100  millisecond  delay.  In- 
formation of  this  nature  is  useful  as  it  in- 
dicates where  in  the  frequency  domain  action 
can  be  taken  to  compensate  for  the  effects  of 
delays. 


The  Prediction  Problem 

As  the  dynamics  processor  iterates,  it 
calculates  a series  of  values  with  the  air- 
craft's present  position  being  represented  by 
the  most  recent  calculations.  In  practice, 
a runninn  estimation  of  a value  for  the  next 
member  of  this  series,  Xn+q , can  be  made  by 
placing  the  most  recent  term,  Xn,  and  the  im- 
mediately preceding  terms,  Xn_ q , Xn-2.  etc., 
into  a prediction  equation  of  the  form: 

xn+l  = Xn  + f (multiple  terms). 

The  most  recent  term,  Xn,  is  given  the  most 
weight  and  a set  of  other  terms  are  used  to 
adjust  Xn  to  approximate  the  future  value. 

The  simplest  case  of  this  approach  is  a linear 
prediction  scheme  where  Xn  and  the  instanta- 
neous slope  of  the  function,  Xn,  are  combined 
such  that  the  predicted  value  is  a linear  ex- 
tension of  the  function  from  point  Xn: 

Xo+ 1 = Xn  + KXn, 

where  K is  the  prediction  span.  This  type  of 
prediction  scheme  has  value  for  flight  simu- 
lators as  both  Xp  and  X_  are  usually  available 
for  each  iteration  of  the  dynamics  processor, 
and  little  processor  time  is  used  to  calculate 
Xn+i . 


An  obvious,  simple,  and  quick  formula- 
tion, linear  prediction  has  been  used  where 
CGI  system  delays  have  existed,  and  has  in- 
volved an  adjustment  of  the  aircraft  position 
values  before  they  are  passed  to  the  graphics 
processor.  A problem  arises  because  KXn  is  a 
product,  so  that  if  either  K or  becomes 
large,  the  adjustment  to  Xn  can  also  become 
large.  The  potential  for  this  condition 
exists  in  simulations  of  responsive,  high- 
performance  aircraft  and/or  where  the  CGI 
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system  has  long  processor  delays.  Usually 
the  processor  delay  remains  constant,  and  the 
use  of  a linear  projection  scheme  results  in 
an  amplification  of  the  high-frequency  re- 
sponses of  trie  simulated  airframe.  For  air- 
craft responses  which  are  low  in  frequency, 
the  predicted  value  of  Xn+-|  does  not  change 
much  from  one  iteration  of  the  dynamics  pro- 
cessor to  the  next,  but  the  amplification  of 
the  hiqh-frequency  responses  produces  values 
for  Xn+]  that  can  fluctuate  quite  a bit, 
causing  a "jitter"  in  the  visual  display. 

Test  pilots  evaluating  simulators  have  found 
the  degree  of  this  jitter  annoying  (Larson 
and  Terry,  1975),  and  when  they  were  allowed 
to  vary  the  temporal  extent  of  the  linear  ex- 
trapolation, they  adjusted  K to  a fraction  of 
the  true  CGI  system  delay. 

This  brings  us  to  the  real  problem  in 
compensating  for  CGI  system  delays,  and  that 
is  the  development  of  a rule  for  limiting 
the  amount  by  which  Xn+i  is  allowed  to  differ 
from  Xn.  If  an  aircraft  parameter  is  allowed 
to  change  too  slowly,  the  simulation  will  ap- 
pear sluaaish  relative  to  the  real  aircraft, 
but  on  the  other  hand,  if  the  rate  of  change 
is  too  high,  the  visual  display  will  have  re- 
duced acceptability  because  of  the  imposed 
jitter.  Since  the  visual  system  delay  is 
relatively  constant  for  a given  system,  we 
have  decided  not  to  limit  the  interval  over 
which  a parameter  will  be  projected,  but  to 
reduce  the  amount  that  a compensation  scheme 
can  amplify  high-frequency  responses  by  pass- 
ing adjusted  parameters  through  a low-pass 
filter.  To  see  if  such  an  adjustment  sequence 
has  any  utility,  we  have  developed  and  evalu- 
ated a delay  compensation  scheme  that  repre- 
sents the  least  complex  example  of  this  ap- 
proach, where  the  extrapolation  for  the  entire 
delay  from  Xn  to  Xn+i  is  linear  and  the  low- 
pass  filter  is  first  order  (i.e.,  has  a 6dB 
per  octave  attenuation  rate  above  the  break 
frequency) . 


The  Present  Effort 

Our  efforts  to  develop  compensations 
for  visual  system  delays  of  the  sort  found  in 
CGI  systems  have  involved  two  approaches.  In 
preliminary  experiments,  we  found  very  little 
decrement  in  the  performance  of  a tracking 
task  in  which  a 200  millisecond  delay  had 
been  inserted.  At  about  this  time,  the 
Cooper,  Harris,  and  Sharkey  study  found  that 
differences  in  performance  measures  such  as 
trials-to-criterion  were  almost  nonexistent 
for  experienced  pilots.  We  suspected  that 
this  result  was  obtained  because  the  experi- 
enced pilots  had  developed  the  ability  to 
quickly  assess  the  dynamics  of  the  system  they 
were  controlling  and  to  change  their  style 
accordingly.  This  feeling  was  reinforced  by 
their  second  experiment  where  differences  of 
measures  of  the  pilots'  control  inputs  were 
found  between  the  delay  and  no-delay  condi- 
tions. We  felt  that  experienced  subjects 
might  not  be  best  for  the  assessment  of  delay 
compensations  developed  for  flight  trainers, 
but  that  they  would  be  ideal  for  the  defini- 
tion of  the  limits  of  piloting  control  for  a 
given  simulator  under  given  conditions.  Thus, 
in  order  to  develop  compensations  for  display 
delays,  we  used  a two-stage  procedure  where 
the  effects  of  experimental  manipulations 
were  tested  using  subjects  familiar  with  the 
control  tasks  and  the  training  usefulness  of 
the  developed  compensation  was  assessed  using 
naive,  untrained  subjects. 

Control  performance  under  a variety  of 
conditions  was  measured  by  having  either 
trained  or  naive  subjects  "fly"  the  system 
depicted  in  Figure  2.  The  compensatory  dis- 
play was  an  oscilloscope  configured  to  re- 
present an  artificial  horizon,  and  the  sub- 
jects entered  their  control  inputs  via  a 
two-axis  spring-centered,  side-arm  control 
stick.  Two  types  of  aircraft,  designated 
Task  1 and  Task  2,  could  be  simulated.  The 
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Figure  2.  Block  Diagram  of  the  Control  System  and  Experimental  Conditions. 
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dynamics  of  Task  1 were  representative  of 
flyina  a general  aviation,  light,  twin- 
engine  craft,  while  Task  2 represented  flying 
a high-performance  jet  interceptor.  For  both 
tasks,  the  subjects  were  asked  to  control  the 
horizon  indicator  so  as  to  maintain  a 
straight-and-level  attitude  in  the  face  of 
mild  turbulence. 


At  the  end  of  each  two-minute  trial, 
measures  of  the  subject's  control  perform- 
ance were  available  from  the  scoring  system. 
Models  of  piloting  control  suggest  that  pi- 
lots adjust  the  size  and  timing  of  their 
control  inputs  to  make  the  system-plus- 
controller  conform  to  certain  requi rements , 
and  our  measures  were  chosen  to  reflect  those 
adjustments  (see  McRuer,  Graham,  Krendel,  and 
Reisener,  1965).  In  the  first  two  experiments 
where  subjects  experienced  with  the  system 
dynamics  were  tested,  the  absolute  system 
error,  the  size  of  the  control  stick  deflec- 
tions, and  their  relative  spectral  power  with- 
in a frequency  band  in  the  pilot  "crossover" 
region  (two  to  six  radians/second)  were  mea- 
sured. These  measures  were  integrated  and 
averaged  over  the  length  of  a trial.  In  the 
third  experiment,  where  naive  subjects  were 
trained,  trials-to-cri terion  was  added  to  the 
measure  set.  Criterion  performance  was  ex- 
pressed in  terms  of  the  displayed  pitch  and 
roll  error  of  the  aircraft,  and,  to  be  con- 
sidered "trainee,"  the  naive  subjects  had  to 
be  able  to  maintain  the  system  pitch  and  roll 
errors  under  1°  and  3°,  respectively,  for  two 
consecutive  trials.  For  our  tasks,  this  re- 
presents a high-per iormance  criterion,  al- 
though the  experienced  subjects  were  able  to 
generate  average  error  well  below  these  limits. 


The  three-position  switch  in  the  error 
feedback  loop  of  Figure  2 indicates  the  con- 
ditions under  which  data  were  collected. 

When  the  switch  is  in  position  1,  the  error 
signals  from  the  dynamics  portion  of  the 
simulator  software  are  immediately  available 
to  the  display,  and  the  only  delay  is  the 
calculation  time  for  the  displayed  values. 

For  our  tasks,  this  time  averaged  to  be  17.5 
milliseconds.  Changing  the  feedback  switch 
to  position  2 allowed  us  to  insert  transport 
delays  into  »he  loop.  Our  simulation  iter- 
ated 20  times  a second,  and  these  delays 
were  progranined  to  be  multiples  of  that  50 
millisecond  period.  While  delays  of  any 
length  could  be  inserted,  we  did  not  examine 
performance  under  any  longer  than  1400  milli- 
seconds. Feedback  switch  position  3 repre- 
sents the  conditions  under  which  we  have 
developed  and  evaluated  compensations  for 
display  delays.  With  this  feedback  loop, 
aircraft  parameters  were  compensated  and  de- 
layed before  they  reached  the  visual  display. 
For  each  experimental  condition,  the  span  of 
prediction  was  equal  to  the  display  delay. 


RESULTS  OF  EXPERIMENTS 
Experiment  1 

Our  first  interest  was  to  see  how  the 
insertion  of  a display  delay  affected  our 
performance  measures.  In  the  first  two  ex- 
periments, subjects'  control  performance  was 
indicated  by  three  scores.  The  error  which 
a subject  produced  was  used  to  reflect  how 
controllable  he  found  the  system,  and  the 
stick  deflection  and  relative  power  scores 
were  used  to  indicate  the  control  technique 
he  used  to  generate  that  error.  Experienced 
subjects  were  used  in  these  two  experiments, 
and  their  data  are  presented  as  individual 
functions  in  the  resulting  graphs.  In  this 
experiment,  performance  under  a variety  of 
delays  was  observed,  and  our  control  measure- 
ments for  Tasks  1 and  2 are  plotted  separate- 
ly in  Figure  3.  Only  measurements  for  roll- 
axis  control  are  presented  as  almost  no  dif- 
-ferences  were  seen  between  delay  and  no-delay 
conditions  for  the  control  of  the  pitch  axis. 

From  the  plots  of  system  error  at  the 
top  of  the  figure,  several  trends  are  evident. 
Controllability  gets  worse  with  long  delays, 
and  good  control  is  harder  for  the  hiyher 
performance  aircraft.  Both  of  these  results 
are  as  one  might  expect.  In  addition,  how- 
ever, our  subjects  displayed  large  and  con- 
sistent individual  differences.  On  both 
tasks,  some  of  the  subjects  generated  up  to 
twice  the  error  that  others  did,  and  the  rank 
order  of  their  error  scores  does  not  change 
from  the  first  task  to  the  second. 


The  stick  deflection  measure  shows  a 
pattern  similar  to  that  obtained  from  the 
error  scores.  Subjects  who  initially  made 
large  stick  deflections  responded  to  small 
delays  by  reducing  the  size  of  their  control 
movements,  but  eventually  larger  and  larger 
control  deflections  were  needed  to  cope  with 
the  longer  delays. 


Our  third  measure,  the  relative  power 
in  the  crossover  region,  shows  striking  dif- 
ferences between  subjects.  Some  subjects 
hardly  changed  the  extent  to  which  they  were 
making  movements  within  this  frequency  region 
while  others  showed  large  differences.  As 
the  delay  inserted  into  Task  1 was  lengthened, 
the  initial  response  of  some  was  to  insert 
more  power  into  the  crossover  band.  They 
later  removed  it  as  the  delay  became  still  lon- 
ger. Our  high-performance  task  did  not  al- 
low subjects  this  flexibility,  and  power 
scores  which  were  high  for  a zero  delay  dis- 
played a steady  decline  with  lengthening 
delays.  Clearly  subjects  differ  in  the  ex- 
tent to  which  a display  delay  affects  the 
accuracy  and  style  with  which  they  control 
even  a simple  flight  simulation. 
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500  1000  1500  500  1000 

DISPLAY  DELAY  (msec)  DISPLAY  DELAY  (msec) 

Experiment  1:  Control  Measures  as  a Function  of  Magnitude  of  Display  Delay 
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Experiment  2 

In  the  second  experiment,  we  varied  the 
point  at  which  the  filter  of  our  compensation 
scheme  began  to  attenuate  the  values  passed 
through  the  third  feedback  path  of  Figure  2. 
This  would  allow  us  to  fix  the  break  freguency 
of  that  filter  so  that  the  training  usefulness 
of  a good  linear  predictor-filter  scheme  could 
be  determined.  Subjects  were  tested  on  the 
control  of  Task  2 for  several  fixed  delays. 
Only  the  data  for  the  400  millisecond  delay 
condition  are  presented  in  Figure  4,  as  they 
are  representative  of  the  results  obtained 
with  both  shorter  and  longer  delays. 


Break  frequencies  for  the  filter  were 
chosen  to  span  the  pilot  crossover  region. 

In  Figure  4,  it  can  be  seen  that  this  system- 
atically affected  the  subjects'  error  scores. 
As  the  break  frequency  was  raised,  the  error 
scores  became  smaller,  up  to  a point.  This 
finding  is  not  surprising  as  subjects  cannot 
null  error  which  they  cannot  see.  In  addi- 
tion, these  functions  appear  to  have  a minimum 
at  about  four  to  five  radians/second,  as  some 
subjects  tended  to  increase  their  error  ubove 
that  reqion.  It  is  our  impression  that  the 
increase  of  system  error  seen  above  the  four 


to  five  radians/second  region  is  a function 
of  the  compulsiveness  with  which  the  subject 
tried  to  null  the  displayed  error.  When  the 
break  frequency  of  the  compensation  filter 
was  set  above  that  region,  some  of  the  "jitter" 
introduced  by  the  linear  prediction  was  dis- 
played, and  subjects  who  attempted  to  null 
the  high-frequency  error  tended  to  add  more 
than  they  removed. 

As  in  the  first  experiment,  the  other 
control  measures  responded  systematically. 

The  stick  deflection  scores  went  through  a 
minimum  between  four  to  six  radians/second, 
and  for  some  subjects,  a high-frequency  in- 
crease was  present.  As  the  compensation  fil- 
ter passed  more  components  of  the  error  signal 
and  the  display  appeared  more  responsive, 
the  subjects  increased  the  power  they  placed 
in  the  crossover  region.  This  occurred  until 
a break  frequency  of  two  to  three  radians/ 
second  was  reached;  beyond  that  point,  the 
crossover  power  remained  relatively  constant. 
The  dotted  lines  of  Figure  4 represent  the 
average  value  of  each  measure  for  the  zero- 
delay  condition.  It  can  be  seen  that  a break 
frequency  can  be  chosen  for  which  the  system 
error,  the  stick  deflection,  and  the  cross- 
over power  scores  are  close  to  their  non- 
delayed  values. 


Figure  4.  Experiment  2;  Control  Measures  for  Task  2 as  a Function  of  Break  Frequency 
of  Compensation  Filter.  Inserted  Delay  Equals  400  Milliseconds. 
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Experiment  3 

In  this  experiment,  we  decided  to  fix 
the  break  frequency  of  the  compensation  fil- 
ter and  determine  how  inexperienced  subjects 
would  acquire  the  control  skill  under  delayed 
or  delayed-and-compensated  conditions.  This 
experiment  was  designed  as  an  acquisition  and 
transfer  study  where  subjects  first  learned 
to  control  the  display  using  the  Task  1 dy- 
namics and  then  were  transferred  to  Task  2. 

A delay  with  or  without  the  linear  prediction 
and  filter  was  inserted  into  Task  1 and  our 
main  interest  was  in  the  training  time  (trials 
to  criterion)  that  the  various  conditions  re- 
quired. Since  we  were  also  interested  in  how 
subjects  would  respond  on  a new  non-delayed 
task,  all  subjects  were  asked  to  perform  at 
least  one  trial  on  the  transfer  task.  This 
task  contained  no  delay  and  was  performed 
after  criterion  was  reached  on  Task  1.  Our 
testing  sessions  had  a twenty-trial  limit  so 
that  they  could  be  easily  scheduled,  and  this 
required  that  subjects  reach  criterion  on 
Task  1 by  at  least  the  nineteenth  trial. 


break  frequency  of  the  compensation  filter  was 
fixed  at  4.5  radians/second,  and  with  this 
filter  setting,  display  delays  of  up  to  400 
milliseconds  were  included  in  the  experiment. 

Clearly  the  amount  of  practice  needed 
to  reach  criterion  performance  increases  as 
lonqer  delays  are  inserted  into  the  feedback 
loop.  It  is  also  evident  that  training  time 
is  increased  at  a lower  rate  when  the  delay 
is  compensated.  Both  plots  seem  to  be  simple 
linear  functions  of  the  magnitude  of  the  dis- 
play delay. 

It  is  possible  that  acquiring  the  control 
skill  under  a particular  delay  condition  might 
encouraae  subjects  to  perform  in  ways  unique 
to  that  delay.  So,  to  test  for  this  possibil- 
ity, we  analyzed  (across  delay  conditions) 
the  control  measures  obtained  on  the  last  ac- 
quisition and  the  first  transfer  trials.  Dif- 
ferences of  these  measurements  on  the  last 
acquisition  trial  would  indicate  whether  sub- 
jects developed  different  styles  of  control 
to  produce  criterion  performance,  and  differ- 
ences on  the  transfer  trial  would  indicate  if 
subjects  trained  under  various  delay  conditions 
would  control  differently  on  a standard  unde- 
layed test.  Our  subjects  were  trained  to  an 
error  score  criterion  so  we  expected  that  the 
measurements  of  system  error  would  not  differ, 
and  they  did  not  on  either  the  acquisition  or 
transfer  trial.  In  addition,  no  differences 
were  seen  across  delayed  or  compensated  condi- 
tions in  the  stick  deflection  or  relative 
power  scores.  Apparently  having  subjects  meet 
a strict  performance  criterion  forced  them  to 
be  homogeneous  in  their  control  style,  and  it 
merely  took  then  longer  to  master  the  skill 
when  a display  delay  was  present.  Using  the 
delay  compensation  reduced  the  training  time 
necessary  for  this  mastery. 

CONCLUSION 


Figure  5.  Experiment  3:  Mean  Trials  to 

Criterion  for  Task  1.  Break  Fre- 
quency of  Compensation  Filter  set 
to  4.5  Radians/Second. 

Figure  5 presents  the  trials-to- 
criterion  data  for  conditions  that  represent 
all  three  positions  of  the  feedback  switch 
in  Figure  2.  The  training  time  necessary  to 
master  our  normal  undelayed  system  is  pre- 
sented as  the  zero-delay  point,  and  the  time 
needed  when  delays  from  50  to  150  milli- 
seconds were  inserted  into  the  feedback  loop 
is  presented  in  the  upper  function  marked 
"delayed."  Conditions  where  the  display 
delay  exceeded  150  milliseconds  were  not 
thorouohly  tested  as  many  subjects  did  not 
reach  criterion  within  the  nineteen-trial 
limit.  The  lower  function  labeled  "compen- 
sated" presents  the  data  collected  using  the 
third  feedback  loop  of  Figure  2.  Here  the 


From  these  experiments,  several  conclu- 
sions can  be  drawn  which  may  affect  the  use 
of  flight  training  simulation  systems  that 
contain  CGI  system  display  delays. 

1.  Addition  of  a low-pass  filter  to 
the  linear  prediction  scheme  appears  to  have 
overcome  the  limitations  of  that  technique 
and  to  have  significantly  increased  its  train- 
inq  usefulness.  Linear  estimation  of  future 
values  of  aircraft  parameters  is  greatly  im- 
proved by  removing  the  "jitter"  associated 
with  that  technique  with  an  appropriate  filter. 

2.  In  flight  training  systems  that 
contain  significant  dead-time  display  delays, 
equivalent  trainee  performance  can  be  produced 
using  fewer  training  trials  if  this  form  of 
delay  compensation  is  used.  Even  with  this 
rather  simple  compensation  approach,  a 40% 
reduction  in  training  time  was  achieved  by 
making  an  adjustment  for  a display  delay. 


The  reduction  may  be  further  increased  by 
tailorinq  the  compensation  scheme  to  a Dartic- 
ular  control  axis  of  the  airframe  or  to  the 
requirements  of  a given  flying  task. 

3.  When  subjects  train  on  one  system 
and  then  qo  to  perform  on  another,  there  is 
always  the  question  of  transfer.  In  the  case 
of  CGI  system  delays,  there  is  the  potential 
that  trainees  will  learn  to  "fly  the  simula- 
tor" and  will  acquire  skills  under  conditions 
of  a display  delay  that  will  produce  negative 
transfer  for  nondelayed  tasks.  Happily,  our 
data  indicated  no  evidence  of  this  tendency, 
and  we  suspect  that  it  was  because  the  sub- 
jects were  trained  to  a performance  standard. 
If  a training  program  allows  a fixed  amount 

of  training  time  where  not  all  of  the  trainees 
may  acquire  the  skill  at  the  same  rate,  nega- 
tive transfer  effects  may  appear. 

4.  The  error  tolerances  that  served  as 
training  criteria  in  our  third  experiment 
were  both  narrow  and  constant  throughout  a 
trial.  For  some  flying  tasks,  such  as  carrier 
approach  or  weapons  delivery,  performance 
tolerances  become  progressively  more  narrow 

as  the  task  progresses.  In  contrast,  tasks 
such  as  formation  flying  or  aerial  refueling 
demand  narrow  performance  tolerances  over  an 
extended  period  of  time.  While  a compensation 
for  CGI  system  delays  is  potentially  useful 
in  all  areas  of  flight  simulation,  we  would 
expect  that  improvements  resulting  from  the 
use  of  such  a scheme  would  be  more  dramatic 
for  tasks  of  the  latter  category. 

5.  The  filtered  linear  projection  ' 
scheme  seemed  successful  enough  to  warrant 
further  work  on  delay  compensations  tailored 
to  flying  tasks.  The  difference  spectra  of 
Cooper,  Harris,  and  Sharkey's  study  appear  to 
have  a 12  decibel  per  octave  high-frequency 
attenuation  rate,  suggesting  that  we  can 
change  our  low-pass  filter  to  a second-order 
one,  allowing  its  break  frequency  to  be  set 
higher  while  still  attenuating  the  high- 
frequency  responses  strongly  enough  to  reduce 
the  display  "jitter"  to  acceptable  levels. 

Such  a change  may  not  affect  measures  such 

as  the  system  error,  but  it  should  be  re- 
flected in  higher  test  or  instructor  pilot 
ratings  of  the  acceptability  of  a simulation. 
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ABSTRACT 

Schedule  constraints,  availability  require- 
ments, functional  equipment  modularity,  and  cost 
are  driving  the  design  of  high-speed  pipeline 
processors.  This  paper  describes  the  evolution 
of  signal  processor  products  for  radar  and  sonar 
trainers  that  has  led  to  the  adoption  of  micro- 
processor controlled  implementations. 

INTRODUCTION  AND  SUMMARY 

In  the  past  decade,  DOD  concepts  for  radar 
and  sonar  training  have  evolved  from  simulation 
to  stimulation  of  actual  operational  equipment.  ^ 
This  has  placed  an  extreme  burden  on  the  signal 
processor  for  speed  and  flexibility.  All  present 
day  trainers  use  digi  tal/analog  pipeline  techniques 
for  signal  processing.  This  paper  describes  the 
evolution  of  radar  and  sonar  signal  processor  de- 
velopments that  have  been  driven  by  cost,  sched- 
ule, and  availability.  Examples  of  design  philos- 
ophy have  been  drawn  from  sonar  training  devices 
14EI9,  14A2,  14E23,  14E25,  14E27,  and  future 
sonar  product  ideas;  and  from  the  USAF  Under- 
graduate Navigator  Training  System  (UNTS)  radar 
trainer,  along  with  future  radar  trainer  ideas. 

In  summary,  microprocessors  are  here  to 
stay,  and  the  total  design  process  has  evolved 
toward  "programmable  logic."  With  the  ten- 
dollar  microprocessor  system  available  today, 
the  "old"  days  of  equipment  design  are  gone. 

REQUIREMENTS  AND  AVAILABILITY  HISTORY 

The  program  requirements  of  cost  and 
schedule  have  caused  the  system  modeling,  al- 
gorithm development,  and  hardware  design  to  be 
performed  in  parallel.  At  the  same  time,  the 
equipment  complexity  has  increased  by  ten  every 
three  to  four  years;  the  availability  has  increased 
from  75  percent  to  95  percent,  with  the  MTTR 
decreasing  from  2 hours  to  20  minutes. 

The  preceding  requirements  have  caused 
the  modularity  of  ftinction  and  hardware,  the 
flexibility,  and  the  built-in  test  capability  to  in- 
crease, until  we  have  the  present  microprocessor- 
controlled  architecture  of  the  14E27  Sonar  Oper- 
ator Trainer.  This  design  philosophy,  using 
seven  functional  subsystems,  allows  the  signal 
processor  control  programs  to  be  loaded  during 
final  checkout,  saving  redesign  and  rebuild  lime. 
Also,  each  microcontroller  is  being  used  auton- 
omously for  signal  processing  control,  built-in 
test,  and  checkout  signal  insertion. 


Because  of  the  short  deliverv  schedule  and 
trainer  complexity,  the  system  definition  and  model- 
ing, the  system  software,  and  the  hardware  are  all 
being  designed  in  parallel.  In  the  past,  this  has 
caused  large  perturbations  in  checkout  because 
three  dependent  areas  were  being  integrated,  with 
each  having  numerous  "bugs  . " This  fact  has 
caused  our  systems  to  evolve  to  their  present 
stages.  Now',  the  hardware  is  built  and  checked 
out  independently  of  the  modeling/software  aid, 
as  these  elements  arc  checked  out,  the  hardware 
may  be  modified  via  "firmware  change.  " Thus, 
the  total  checkout/acceptance  test  cost  is  greatly 
reduced. 

A side  benefit  is  that  the  microprocessor 
controller  has  led  to  more  general-purpose  cards 
and  common  design,  which  reduces  the  design 
time  and  leads  to  system  commonality  and  easier 
sparing. 

GENERAL  SYSTEM  ARCHITECTURE 

The  microprocessor  controller  is  a program- 
mable device  designed  to  provide  a flexible  input/ 
output  (I/O)  capability  with  desired  stimulus,  con- 
trol , and  monitor  requirements  for  a variety  of 
digital  systems.  With  these  capabilities,  the  con- 
troller can  be  built  into  large  digital  systems  to 
perform  three  basic  functions  of  primary  concern. 
First,  the  controller  can  provide  the  control  func- 
tion for  special-purpose  hardware.  This  is  partic- 
ularly significant  since  it  means  that,  in  general, 
the  total  logic  hardware  design  can  be  reduced  to 
the  design  of  generic  type  cards;  i.e.  , data  reg- 
isters, data  selectors,  counters,  adders,  multi- 
pliers, etc.  The  timing  clocks  and  mode  controls 
arc  then  furnished  by  the  controller  under  pro- 
gram control.  Second,  the  controller  can  sub- 
stitute an  idealized  stimulus  for  test  purpose  and 
monitor  results  to  perform  automatic  built-in  test- 
ing. Third , the  controller  can  be  programmed  to 
provide  an  idealized  stimulus  as  desired  to  aid 
"in-system"  signal  tracing,  which  is  particularly 
valuable  during  initial  system  checkout. 

The  controller  architecture  is  similar  to 
that  of  general-purpose  computers  in  that  the  CPU 
consists  of  a central  control  unit,  arithmetic/ 
logic  unit,  and  program  memory.  The  central 
control  unit  itself  differs  radically,  though,  from 
the  normal  computer  control  unit,  in  that  the  in- 
struction set  is  allowed  to  operate  directly  on 
various  special-purpose  I/O  registers.  Since  the 
instruction  execution  time  is  very  fast,  this  allows 
a high-speed  dynamic  stimulus  capability  for  ex- 
ternal control.  Special  consideration  is  given  to 
provide  flexibility  in  the  co  oiler  for  receiving 
data  from  the  external  system  for  monitoring 
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purposes.  Also,  the  capability  exists  to  test  the 
level  of  input  control  lines  that  can  be  used  as 
flags  to  cause  program  branches. 

In  a system  that  is  to  use  a microprocessor 
controller,  the  system  partition  and  design  pro- 
cess is  as  s it  own  in  Figure  1.  The  key  to  the  whole 
procedure  is  that  the  system  is  partitioned  so  that 
each  subsystem  with  microprocessor  controller 
can  be  packaged  in  one  card  rack.  This  card  rack 
then  has  a small  set  of  inputs  and  outputs  and  per- 
forms a total  system  function. 

The  microprocessor  controller  evolution  is 
now  leading  to  more  generic  designs,  autonomous 
card  racks  with  built-in  power  supplies,  totally 
ground-noise  isolated  bv  optic  communication 
links,  bit-sliee  microprocessor  applications,  and 
increases  in  the  level  of  built-in  test  functions. 

CONTKOLLE  K A HC'HiTKCTU  RE 

The  general-purpose  controller  is  shown  in 
Figure  2.  The  controller  Is  a 16-bit  machine 
capable  of  executing  a variety  of  instructions,  in- 
cluding data  transfer,  arithmetic,  logic,  and 
branch  type  instructions.  Special  features  include 
(1)  the  ability  to  perform  branch  instructions,  de- 
pendent upon  a wide  selection  of  internal  and  ex- 
ternally generated  flags,  (2)  the  ability  to  condition 
input  data  registers  to  meet  the  requirements  of 
the  external  device,  (3)  the  ability  to  gate  system 
clocks  to  the  external  system,  and  (4)  the  ability 
to  operate  from  either  RAM  or  ROM  program 
memory.  The  instruction  execution  time  depends 
on  the  master  clock.  With  the  master  clock  in  the 
range  of  5 to  10  mHz,  all  Instructions  (except 
transfer  immediate)  take  two  times  the  master 
clock  period;  while  the  transfer  immediate  instruc- 
tion may  be  wired  to  take  one  times  the  master 
clock  period.  Below  a 5 mHz,  all  instructions  are 
wired  to  take  the  master  clock  period  (e.  g.,  5 mHz 
clock  given  200  nanosecond  Instruction  time). 


The  input  data  registers  are  l(i-bit  parallel 
registers  that  can  be  added  in  increments  of  three 
registers,  to  a maximum  of  six.  Under  program 
control,  It  is  possible  to  condition  each  of  these 
registers  uniquely  to  load  the  register,  using 
either  an  externally  generated  clock  or  an  inter- 
nally generated  constant  clock.  Program  instruc- 
tions arc  provided,  allowing  the  controller  to  use 
the  data  from  the  input  register. 

The  four  general-purpose  save  registers 
serve  as  temporary  holding  registers  that  can  be 
used  as  desired  during  program  execution.  Save 
register  zero  also  acts  as  one  side  of  the  compare 
function  for  comparative  branch  tvpe  instructions. 

The  ALU  functions  to  perform  arithmetic/ 
logic  functions  on  selected  accumulators,  output 
registers,  or  the  compare  register.  The  results 
are  then  stored  back  into  the  designated  accumu- 
lator, output  register,  or  compare  register.  All 
arithmetic  operations  assume  positive  data  only, 
and  overflow/underflow  conditions  are  ignored. 

The  accumulation  HAM  provides  16  general- 
purpose  accumulators  that  are  directly  addressed 
by  the  program;  i.  e. , specific  operations  can  be 
performed  on  designated  accumulators.  One  ac- 
cumulator is  used  for  compare  operations;  six 
are  used  to  control  output  registers,  and  the  re- 
maining nine  function  as  general-purpose  ac- 
cumulators. 

The  output  data  registers  are  16-bit  parallel 
registers  that  can  be  expanded  to  a maximum  of 
six  registers,  in  increments  of  three.  The  con- 
tents of  the  output  data  registers  can  be  used  di- 
rectly in  arithmetic/logic  or  shift  instructions, 
since  they  are  copied  in  equivalent  accumulators. 

The  external  clock  control  provides  a means 
whereby  the  controller  internal  clock,  or  clock 
divided  by  two,  can  be  gated  to  the  external  device 


Figure  1.  Development  Process 
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Figure  2.  General-Purpose  Controller  Basic  Block  Diagram 


under  program  control.  This  is  particularly  use- 
ful in  pipeline  type  processing. 

The  flag  select/test  logic  selects  a flag  for  a 
conditional  branch  type  instruction,  ind,  if  the  ap- 
propriate condition  of  that  flag  exists,  it  will 
cause  a program  branch  to  the  specified  program 
address. 

The  program  counter  provides  the  program 
address  to  the  program  memory.  Under  normal 
operation,  instructions  are  executed  sequentially 
until  a program  branch  causes  the  program  to 
alter  this  sequence. 

The  program  memory  contains  the  coded 
program  in  either  ItAM  or  ROM  units;  primarily, 
RAM  for  checkout  and  ROM  for  production  units. 
The  program  memory  is  32  bits  wide,  consisting 
basically  of  a 16-bit  instruction  field  and  a 16-bit 
data  field  for  use  in  immediate  instructions;  i.  e. , 
program  memory  provides  data  for  transfer  or 
specified  operation  on  a designated  register. 

The  instruction  register  stores  the  instruc- 
tion accessed  from  the  program  memory  such  that, 
during  its  execution,  the  next  instruction  can  be 
simultaneously  accessed  from  the  program 
memory. 

The  instruction  decode  logic  determines  the 
type  of  operation  to  be  performed  It  decodes 
which  source  register  will  drive  that  data  bus, 
which  destination  register  to  load,  which  flag  is 
selected,  and  any  other  control  signals  required. 

The  control  panel  input  register  is  applicable 
only  if  the  control  panel  is  provided.  It  allows 
the  operator  to  enter  data  from  the  control  panel 
to  the  controller. 

The  optional  scratchpad  memory  provides 
temporary  storage  of  up  to  IK  words  of  data,  and 
is  organized  on  a single  logic  board. 

The  optional  control  panel  display  register 
allows  the  contents  of  any  source  register  to  be 
monitored  by  transferring  the  contents  of  the  de- 
sired register  to  the  display  register,  which  is 
then  displayed  on  the  control  panel. 

MICROPROCESSOR  CONTROLLER  APPLICATION 

As  an  example  of  the  microprocessor  con- 
troller applied  to  pipeline  signal  processing,  the 


target  processor  for  the  14E27  trainer  will  be 
explained.  Figure  3 is  a block  diagram  of  the 
14E27  Pipeline  Signal  Function  Generator.  It  is 
set  up  and  operated  by  an  H-716  Minicomputer, 
and  stimulates  an  SQR-17  Spectrum  Analyzer. 

For  our  example,  we  will  focus  in  on  the 
Narrowband  Target  Processor,  with  the  under- 
standing that  each  of  the  blocks  in  Figure  3 also 
contains  a microprocessor  controller,  anti  oper- 
ates in  a similar  manner.  Figure  4 is  a detailed 
block  diagram  of  the  Target  Processor.  At  first 
glance  it  might  appear  very  complex,  but  if  one 
follows  the  input  signals  across  the  page,  one  can 
see  the  mathematical  operations  performed  on  the 
data.  The  unusual  features  of  the  block  diagram 
are  the  General-Purpose  Controller  (micro- 
processor) in  the  lower  left-hand  corner  (in  past 
systems  this  has  been  a hardwired  controller), 
and  the  insertion/sample  test  data  points  shown  at 
the  GPC  and  throughout  the  block  diagram.  Fig- 
ure 5 shows  a flow  chart  of  the  Microprocessor 
Controller  and  gives  an  indication  of  its  opera- 
tion. The  Operational  Program  uses  84  PROM 
locations  and  the  Auto-Test  Program  uses  360 
PROM  locations. 

Also  during  checkout,  programs  were 
written  to  exercise  data  paths  in  a looping  test  so 
that  a logic  analyzer  and  scope  could  be  used  to 
debug  the  hardware. 

A note  of  caution;  at  the  beginning,  a total 
loop  auto  test  was  tried  for  a while  and  failed  be- 
cause of  diverse  and  interactive  errors.  Finally, 
small  portions  of  the  circuitry  were  exercised, 
starting  with  the  inputs  and  working  toward  the 
output.  In  this  systematic  manner,  the  whole  unit 
was  checked  out  in  a short  period  of  time. 

The  microprocessor  is  destined  to  be  used 
in  all  future  trainers  and  DOD  equipment.  This 
paper  has  shown  that  the  evolution  of  the  Pipeline 
Signal  Processor  (because  of  cost,  schedule,  and 
availability)  has  been  driven  to  a microprocessor 
controller  that  can  provide  operational  control, 
perform  built-in  test,  and  aid  in  the  checkout  of 
the  unit. 
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Figure  3.  Device  14E27  Function  Generator  Basic  Block  Diagram 


AUTO  TEST 
360  PROM  WORDS 


OPERATIONAL  PROGRAM 
84  PROM  WORDS 


Figure  5.  Firmware  Flow  Diagram 
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INTRODUCTION 

The  conventional  use  of  simulators  for 
flight  research,  be  they  moving  base  or  fixed 
base,  has  involved  an  attempt  at  reproducing 
some  aspects  of  face  validity  without  deter- 
mining either  the  requirements  for  such 
validity  or  the  aspects  of  the  simulation 
which  are  really  important  in  either  research 
or  training.  Limited  knowledge  of  the  physio- 
logical sensing  mechanisms,  especially  for 
the  perception  of  motion,  has  forced  upon  the 
simulation  community  an  acceptance  of  "expert 
pilots’"  opinions  as  the  sine  qua  non  of 
simulator  design  and  acceptability.  Yet, 
very  rarely  do  two  pilots  agree  on  a' 1 aspects 
of  a simulator's  fidelity.  With  the  develop- 
ment of  better  models  of  physiological  pro- 
cessing of  sensory  signals  and  a framework 
for  their  use  based  upon  modern  control 
theory,  we  are  now  in  a position  to  improve 
upon  this  situation  for  simulator  planning. 
This  paper  describes  a number  of  parallel 
research  efforts  (Ref.  1), which  are  being 
carried  out  by  the  Air  Force  Human  Resources 
Laboratory,  to  (1)  improve  the  basic  physio- 
logical subsystems  descriptions  and  (2) 
integrate  the..*-  subsystems  into  a model  for 
simulator  planning.  It  is  anticipated  that 
such  a model  will  not  only  aid  in  the  planning 
of  simulators  but  also  possibly  become  a major 
portion  of  the  simulator's  drive  logic. 


CONVENTIONAL  VIEW 

Current  simulation  methodology  is  based 
upon  an  open  loop  strategy  a«f  depicted  in 
Figure  1.  The  assessment,  other  than  through 
pilot  opinion,  is  typically  on  the  basis  of 
closed-loop  control.  Does  the  pilot  "do  the 
same  thing"  in  the  simulator  as  he  does  in 
the  real  aircraft?  Since  the  adaptability 
of  the  human  pilot  makes  such  closed-loop 
performance  criteria  relatively  insensitive 
to  the  simulation  fidelity,  they  are  not 
particularly  convincing.  The  conventional 
method  is  to  simulate  the  aircraft's  flight 
equations  in  real-time  on  a computer  and  to 
use  the  aircraft  state  variables  to  drive 
separately  a visual  display,  instruments, 
motion  platform,  sound  system,  and  any 
G-cuing  devices.  Each  of  these  systems  is 
commanded  to  reproduce  as  accurately  as 
possible  the  sights,  sounds,  and  feel  pro- 
duced by  the  corresponding  aircraft  situa- 
tion. The  assumption  is  that  the  pilot 
will  be  provided  the  best  overall  simulation 
if  he  is  presented  with  individual  sensory 
modalities,  each  of  which  is  tuned  to 
reproduce  the  real  world  situation  as  best 
as  possible  within  the  limitations  of  cost 
and  mechanical  drives.  Relatively  little 
effort  has  been  paid  to  the  question  of  how 
to  best  compromise  any  one  of  the  simulation 
modalities  or  how  to  best  combine  the 
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Figure-  1.  Current  Simulation  Design  Method  Based  on  Duplicating  Aircraft  Motion 
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different  sensory  input  simulations  to  produce 
an  overall  effect  which  is  better  than  that 
produced  by  working  with  them  individually. 

For  example,  the  question  of  the  influence 
of  moving  visual  fields  on  the  requirements 
for  low  frequency  cab  motion  drives  is  an 
important  factor,  one  which  Is  not  addressed 
by  this  design  philosophy.  A limitation  of 
the  conventional  design  philosophy  is,  in 
our  opinion,  the  concentration  on  faithfully 
reproducing  aircraft  motions  and  force  rather 
than  reproducing  pilot  sensations. 

THE  "PILOT  SENSORY  MODEL"  APPROACH 
TO  SIMULATOR  DESIGN 

If  we  assume  that  the  goal  of  simulation 
is  to  produce  the  same  sensations  of  motion 
and  force  in  the  pilot  during  the  simulation 
as  he  would  receive  during  actual  flight, 
then  one  is  led  to  a design  configuration 
illustrated  in  Figure  2.  Underlying  this 
philosophy  is  the  existence  of  an  adequate 
model,  the  "multisensory  motion  sensing 
model  of  pilots"  which  predicts,  in  at  least 
a statistical  sense,  the  pilot's  perception 
of  spatial  orientation  based  upon  defined 
visual,  motion,  and  force  inputs  occurring 
individually  or  in  combination.  This  model 
is  applied  to  the  actual  aircraft  situation 
being  investigated  to  determine  what  the 
pilot's  perceptions  would  be  in  the  aircraft. 
It  is  then  also  applied  to  the  proposed 
simulator  output  to  predict  what  the  errors 
in  perception  would  be.  Simulator  visual, 
motion,  and  G-cuing  drive  algorithms  may  then 
be  designed  to  produce  simulator  output 
within  the  mechanical  limits  of  the  equipment 
which  will  minimize  this  error  in  perceived 


orientation.  Clearly,  the  major  weakness  in 
this  design  philosophy  is  the  lack  of  ade- 
quate multisensory  motion  sensing  models. 
Attempts  to  remedy  this  situation  are  a part 
of  our  current  research.  As  Indicated  in 
Figure  3,  we  are  treating  perceived  orienta- 
tion as  an  optimum  estimator  problem  in 
which  a Kalman  filter  estimates  the  perceived 
orientation  based  on  the  outputs  of  the 
various  sensory  systems  each  of  which  is 
assumed  to  have  a characteristic  dynamic 
response  and  associated  additive  noise.  The 
models  for  sensation  of  angular  velocity 
attributable  to  the  vestibular  senses  are 
relatively  well  developed.  Even  complex 
motions,  involving  possibly  contradictory 
information  presented  by  the  otoliths  and 
semicircular  canals,  can  be  treated  by 
such  models  (Ref.  2).  Similar  models  for 
foveal  visual  inputs,  which  basically  relate 
to  instrument  readings,  have  been  developed 
and  require  only  application  of  the  appro- 
priate fraction  of  attention  divided  to  each 
instrument  to  be  applied  (Ref.  3).  Periph- 
eral visual  information  which  leads  to 
perceived  changes  in  orientation  or  angular 
rate  are  being  developed  on  the  basis  of 
work  on  visually  induced  motion  (Ref.  4). 

The  greatest  area  of  uncertainty  is  in  the 
processing  of  tactile  and  proprioceptive 
information.  Data  from  these  systems  will 
be  most  important  in  determining  stimulation 
and  drive  requirements  for  G-cuing  devices, 
including  G-seats,  G-suits,  and  seat  shaker 
systems.  We  are  currentlv  using  both 
psychophysical  and  physiological  data  to  try 
to  develop  both  the  high  and  low  frequency 
characteristics  of  these  sensory  modalities 
and  to  achieve  appropriate  noise  levels  so 
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that  they  may  be  integrated  in  the  Kalman 
filter  and  blended  with  the  other  sensory 
modalities.  In  addition,  it  must,  of  course, 
be  recognized  that  the  pilot  is  engaged  in 
an  active  flying  task.  Consequently,  some 
means  must  be  made  of  placing  into  the  con- 
text of  this  model  the  pilot's  expectation  of 
aircraft  response  on  the  basis  of  applied 
demands. 


It  is  our  hope  that  the  eventual  applica- 
tion of  pilot  models  based  upon  underlying 
physiological  mechanisms  will  provide  a 
rational  way  of  simulator  planning  to  provide 
the  pilot  with  the  minimum  amount  of  simulator 
hardware  required  to  give  an  adequate  repro- 
duction of  orientation  sensations  for  either 
training  or  research  simulators.  Furthermore, 
it  is  our  goal  to  incorporate  such  a motion 
sensing  model  into  the  conventional  simulator 
drive  algorithm  to  facilitate  the  optimum 
performance  of  the  device.  Results  from  the 
modeling  effort  are  anticipated  in  early  1977. 
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PILOT  PERFORMANCE  MEASUREMENT  SYSTEM  FOB  THE 
A -7  NIGHT  CARRIER  LANDING  TRAINER  (NCLT) 

THOMAS  J.  KLEIN  and  CAR].  E.  . iATTLAGE 

Vought  Corporation  - Systems  Division 


ABSTRACT 

Vought  Corporation  - Systems  Division  is 
under  contract  to  NAVAIR  (4131)  to  develop  a 
pilot  performance  measurement  system  that  will 
aid  the  Landing  Signal  Officer  (LSO)  during 
training  in  the  A-7  Night  Carrier  Landing 
Trainer  (NCLT).  This  paper  discusses  (a)  how 
parametric  performance  data  (lineup  and  glide- 
slope  error,  sink  rate,  airspeed,  etc.)  are 
collected  on  categories  1 and  II  A-7  replace- 
ment pilots  during  NCLT  and  night  carrier 
qualifications,  (b)  how  the  data  will  be 
statistically  analyzed  to  determine  those 
factors  that  indicate  student  progress,  (c) 
how  these  factors  will  be  combined  into  a 
single  predictive  index,  and  (d)  how  the 
program  products  can  be  applied  as  training 
aids.  Finally,  the  results  of  preliminary 
analyses  are  presented. 

INTRODUCTION 

The  Vought  Corporation  - Systems  Division 
is  conducting  a Might  Carrier  Landing  Trainer 
data  collection  program  for  the  Naval  Air 
Systems  Command  (AIR-4131)  under  Contract  No. 
N00019-73A-0010.  The  objective  of  the  program 
is  to  develop  valid  computer-scored  measures 
of  student  pilot  progress  in  the  NCLT.  These 
measures  are  also  to  be  explored  for  their 
predictive  value  in  the  field  carrier  landing 
practice  (FCLP)  and  the  carrier  qualification 
(CAFQUAL)  environments.  The  resultant  mea- 
sures are  to  be  incorporated  into  a student 
pilot  trend  analysis  summary  printout  as  a 
training  aid  to  the  Landing,  Signal  Officer 
(LSO)  community. 


relationship  between  NCLT  and  CARQUAL  perfor- 
mance exists  and  that  a more  in-depth  analyti- 
cal effort  should  be  undertaken  to  validate 
the  system  and  to  develop  the  desired  program 
products . 

Figure  1 is  a simplified  overview  of 
data  collection,  data  analysis  and  program 
product  relationship.  Shown  are  the  data 
from  two  samples  (development  and  validation) 
of  30  replacement  pilots  (RP's)  going  through 
the  NCLT-FCLP-CARQUAL  cycle.  These  data  are 
then  shown  to  be  analyzed  for  the  ultimate 
development  of  the  program  products  - a grad- 
ing system  and  an  instructional  strategy 
(training  aid).  Data  collection,  analyses 
and  validation,  and  training  aid  development 
are  discussed  in  later  paragraphs. 


Figure  1.  Overview  of  Program 


Vought  is  currently  contracted  to,  (a) 
develop  and  install  an  automatic  data  collec- 
tion system  in  the  NCLT,  (b)  collect  quanti- 
tative NCLT  and  CARQUAL  performance  data  on 
60  A-7  replacement  pilots,  (c)  collect 
questionnaire  data  and  qualitative  LSO  student 
performance  conments  and  records  for  the  60 
replacement  pilots,  (d)  perform  preliminary 
analyses  on  these  data,  and  (e)  recommend 
NCLT  training,  aids. 

Preliminary  performance  measures,  grad- 
ing boundaries,  and  training  aid  formats  have 
been  developed  for  the  data  collection  system. 
At  this  time,  the  data  collection  system  has 
been  implemented  at  NAS  LeKoore,  California, 
and  is  operational.  The  data  collection 
effort  is  complete.  The  results  of  prelimi- 
nary analyses  Indicate  a predictive 


Preliminary  Performance  Measures 

The  initial  candidate  performance  mea- 
sures selected  included  lateral  and  vertical 
error,  sink  rate,  and  airspeed  at  specified 
intervals  (3/It,  l/P,  l/4,  l/R  NM  and  at  the 
ramp).  These  measures  were  selected  because 
they  are  the  only  measures  tiiat  can  be  col- 
lected through  the  carrier  ACLS  SFW-42  radar 
system.  However,  additional  performance  mea- 
sures were  selected  for  collection  in  the 
NCLT.  These  include  glideslope,  lineup,  fuel 
flow,  angle  of  attack  and  pitch  attitude  as 
well  as  velocity  vector  and  a work  response 
index.  Hie  measurement  system  is  designed  to 
continuously  sample  these  parameters  from 
1 l/4  NM  to  the  ramp,  and  to  print  out  means 
and  variances  for  each  of  six  selected  seg- 
ments. Both  absolute  error  data,  and  error 
data  weighted  by  range,  are  considered. 
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Weight  inf,  t y range  emphasizes  the  criticality 
of  errors  when  "close  in."  Mean  percent  time 

in  tolerance  for  the  first  five  of  these 
parameters  are  also  collected  across  each  of 
the  six  segments. 

The  ;:CIT  data  llection  system  printout 
is  shown  in  Figure  2.  The  figure  shows 


that  the  printout  includes  initial  an.!  ter- 
minal conditions,  record-keeping  data,  and 
computed  grades  i'or  lineup,  p.lirteslope,  angle 
of  attack,  fuel  flow  and  pitch  attitude. 

These  grades  (0  to  h scale)  are  based  on  pre- 
liminary performance  boundaries  established 
early  in  the  program  and  are  implemented  in 
the  current  data  collection  system. 


Figure  2.  NCLT  Data  Printout  Format 
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IT'']  . in-ir;  performance  Donna. tr ios 

! "re limitary  boundaries/standards  were 
identified  fron  existing  operational  date  and 
fro:  handling  qualities  characteristics  of 
the  A-7S.  These  were  refined  through  evalu- 
ation by  LSO’s  and  are  currently  used  as  the 
initial  set  of  standards. 

At  this  tine,  preliminary  boundaries 
have  been  further  refined  for  glideslope, 
lineup,  angle  of  attack,  fuel  flow,  and  pitch 
attitude.  This  modification  was  based  on  the 
results  of  a group  of  seven  LSO’s  flying  202 
approaches  in  the  NCLT.  The  approaches  were 
grouped  by  LSO  grade  (0.0  to  U.O)  and  com- 
pared with  the  total  sample  of  nixed  grades. 
Figure  3 illustrates  the  rationale  for  bounda- 
ries that  are  being  established  for  all  per- 
formance measures.  This  example  (based  on 
glideslope  performance  data)  shows  how  upper 


Figure  3-  Sample  Vertical 
Error  Boundary  Rationale 


arid  lower  limits  for  0.0  through  U.O  grades 
are  established  at  six  points  from  1 l/U  NM 
out  to  the  Number  1 wire.  "Acceptable”  upper 
and  lower  limits  are  being  established  by 
examining  three  types  of  data:  (a)  one  stan- 

dard deviation  from  the  mean  error  at  the 
windows"  for  all  approaches  (class  average), 
(b)  the  moan  values  for  a minimally  acceptable 
approach  (grade  of  2.0)  and  (c)  the  region 
within  which  LSO  comments  do  not  occur. 

The  initial  boundary  sets  will  be  updated 
and  refined,  subsequent  to  the  completion  of 
the  data  analysis  phase. 

DATA  COLLECTION 

Initial  data  collection  activities  in- 
volved testing  a sample  of  7 LSO's  from  NAS 
LoMoore  (VA-122,  VA-125)  and  NAS  Cecil  Field 
(VA-17U)  in  the  NCLT  to  refine  the  preliminary 
performance  standards.  This  uata  collection 
effort  was  conducted  at  NAS  LeMoore, 


California.  NCLT  predictor  and  criterion 
data  were  collected  for  the  LSC  sample  during 
NCLT  sin'ilation  as  currently  defined  in  the 
A-7  Replacement  Pilot  NCLT  syllabus. 

Iri  addition  to  flying  the  NCLT,  the  LSO's 
ha've  participated  in  the  program  in  other 
ways:  first,  they  were  consulted  to  provide 

subjective  information  relevant  to  the  ration- 
ale behind  their  descriptive  comments  and 
symbology  when  evaluating  a student  night 
carrier  landing  pass.  This  information  aided 
in  the  formulation  of  a rationale  for  develop- 
ment of  a computerized  trend  analysis  system. 
Next,  the  LSO's  again  flew  the  NCLT  to  pro- 
vide additional  flight  data  for  debugging,  the 
initial  computerized  procedures  for  assessing 
trends  and  assigning  grades.  Finally,  the 
LSO's  continue  to  monitor  and  assess  both 
NCLT  and  CARQUAL  performance  to  provide 
grades . 

Data  from  two  separate  samples  of  30  A-7 
category  I and  category  II  replacement  pilots 
(RP's)  was  collected  in  the  NCIT  and  during 
CARQUAL  to  provide  a basis  for  development  of 
the  pilot  performance  measurement  system 
products.  Syllabus  variations  have  been 
minimized  by  utilizing,  for  analytical  p:rr- 
poses,  only  those  data  applicable  to  the 
relatively  standardized  approaches  flown 
during  NCLT  syllabus  periods  two  through  six. 

CARQUAL  data  collection  efforts  to  date 
included  cruises  aboard  the  USS  Enterprise, 

USS  Saratoga,  USS  Uimitz,  USS  Coral  Sea,  and 
the  USS  Kitty  Hawk.  Data  obtained  aboard 
these  carriers  included  LSO  comments  and  ana- 
log traces  from  the  carriers'  SRJ-U2  equip- 
ment. These  traces  include  ranges,  ship  heave 
and  pitch,  vertical  error,  lateral  error,  air- 
speed, sink  rate  and  closure  speed  for  each 
pass.  Figure  it  is  a partial  sample  of  a 
SFN-it2  strip  chart  recording  for  a carrier 
approach . 


Figure  it.  Sample  SRN-U2 
Strip  Chart  Record 
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SJK-lt£?  data  will  be  converted  to  digital 
form  for  comparison  with  similar  NCLT  data. 

LSO  comment  data  from  both  the  NCLT  and  the 
CARQUAL  environments  will  be  used  to  develop 
a trend  analysis  sunmary  as  a training  aid. 
This  training  aid  is  discussed  in  a later 
paragraph . 

DATA  ANALYSIS  AND  VALIDATION 

The  first  analytical  effort  will  involve 
development  of  normative  performance  (norms) 
for  each  independent  variable  using  NCLT  data 
only.  Statistical  studies  will  examine 
function  distribution  variances  and  means  to 
determine  those  measures  which  are  most 
descriptive  of  techniques  resulting  in  good 
performance.  It  is  anticipated  that  for  some 
of  the  measures,  mean  values  may  be  the  most 
descriptive  of  normative  performance.  For 
others,  such  statistics  as  maximum  deviation 
o:  percent  time  within  tolerance  may  be  more 
appropriate  to  predict  the  number  of  periods 
(passes)  required  by  a pilot  to  reach  the 
criterion  performance  level-  as  set  by  the 
LSO's  for  NCLT  training.  The  data  for  making 
these  predictions  will  be  based  on  pilot 
flight  histories  and  from  NCLT  data. 

Normative  performance  values  will  also 
be  established  for  final  performance  measures. 
These  normative  performance  values  will  be 
determined  for  each  of  the  two  RP  samples 
of  30  students,  and  for  the  combined  sample 
(60  students).  These  data  (mean  errors  and 
standard  deviations)  will  be  plotted  for  the 
five  parameters  that  comprise  the  NCLT  grading 
system  (e.g.,  lineup,  glideslope,  angle  of 
attack,  fuel  flow  and  pitch  attitude)  at  each 
LSO  grade  level.  Hie  plots  will  be  used  to 
establish  criteria  for  grading  the  parameters. 
Figure  5 is  an  example  of  this  type  of  plot. 


Figure  5.  Parameter  Grading  Criteria 


Data  for  the  example  was  extracted  from 
the  LSO  sample  that  was  collected  and 'analyzed 
early  in  the  program.  Grading  boundaries  will 


be  adjusted  such  that  the  parameter  grades 
"best  fit"  the  overall  LSO  grade.  Hie  initial 
boundaries  (those  currently  in  the  NCLT)  will 
be  adjusted  during  further  data  analysis 
efforts . 

Subsequent  to  establishment  of  final 
performance  norms  for  each  of  the  performance 
measures,  multiple  correlation  techniques 
employing  a linear  regression  model  will  be 
used  to  account  for  the  variance  in  1130  scores 
and  Landing  Performance  Scores  (LPS).  Hie  LPS 
assesses  the  quality  of  a landing  on  the  basis 
of  terminal  conditions  (wire  no.,  etc.).  The 
objective  is  to  determine  the  equation  which, 
according  to  a defined  criterion,  best  des- 
cribes the  dependent  variable  (LSO  grade  or 
LPS)  in  terms  of  those  independent  (predictor) 
variables  that  the  analysis  shows  are  factors 
that  influence  the  LSO's  evaluation  of  the 
quality  of  a night  carrier  approach. 

It  is  planned  to  employ  the  Efroymson 
method  (Efroymson,  i960  and  Rosenthal,  1966) 
which  combines  the  forward  selection  (step- 
up)  and  the  backward  elimination  (step-down) 
procedures  with  a check  for  co-linearity. 

Hie  essential  idea  of  the  Efroymson  method  is, 
of  those  variables  that  were  in  the  regression 
equation  at  the  last  step,  delete  that  vari- 
able which  gives  the  "minimum  nonsignificant 
(at  a specified  F-vaiue j increase  to  the 
residual  sums  of  squares"  and  where  the 
multiple  correlation  coefficient  is  not 
significantly  large.  If  no  variable  is 
deleted  in  the  first  stage,  those  variables 
that  were  not  in  the  regression  at  the  last 
step  are  inspected.  Among  the  variables, 
that  variable  which  gives  the  maximum  signifi- 
cant (judged  by  the  F-test)  decrease  to  the 
residual  sums  of  squares  is  added.  If  no 
such  variable  exists,  the  procedure  is 
finished  and  the  "best"  equation  has  been 
determined. 

Data  collected  during  the  program  will 
be  employed  in  the  development  of  the  final 
performance  measurement  system  for  the  NCLT, 
based  on  both  CARQUAL  and  NCLT  variables.  An 
intercorrelation  matrix  will  be  computed  on 
each  of  the  sample  groups  of  RP's  to  modify 
the  preliminary  NCLT  performance  standards 
and  grading  criteria  such  that  they  are  con- 
sistent with  their  CARQUAL  counterparts. 

NCLT  predictor  scores  and  correlations  will 
be  redetermined  in  these  cases  where  accept- 
able performance  envelopes  have  been  modified. 
Intercorrelations  between  segments  will  also 
be  presented. 

Results  of  the  regression  equation 
developed  for  the  first  sample  group  of  RP's 
will  be  evaluated  for  effectiveness  by  using 
predictor  data  in  the  regression  equation 
and  comparing  predicted  criterion  data  to 
NCLT  and  CARQUAL  criterion  data. 
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A double  cross-validation  design  will  be 
employed;  that  is,  the  data  collected  on  the 
second  RP  sample  will  be  used  to  validate  the 
regression  equations  developed  on  the  first 
RP  sample,  and  the  data  collected  for  the 
first  RP  sample  will  be  used  to  validate  th" 
repression  equations  developed  for  the  second 
PP  sample.  The  final  repression  equations, 
grading  and  prediction  system,  performance 
standards,,  and  trend  analysis  forms  will  be 
based  on  ail  60  pilot  subjects  utilising 
both  NCLT  and  CAR^UAL  data.  The  cross  - 
correlation  analysis  will  be  tested  for 
residual  or  second  order  effects. 

Analysis  of  LSO  Comment  Data 

An  important  feature  of  the  NCLT  train- 
inr  aid  to  be  developed  during  this  program 
will  be  automatic  emulation  of  LSO  qualitative 
comments  whenever  pei*formance  does  not  meet 
specified  criteria.  To  establish  these 
criteria,  both  HCLT  and  CARQUAL  data  will  be 
analyzed.  For  example,  if  during  CAPy’JAL, 
the  LSO  records  the  comment,  "high  in  the 
middle"  (HIM),  the  analog  trace  will  be 
examined  for  a peak  in  glideslope  error.  Both 
the  range  of  this  peak  and  the  corresponding 
magnitude  of  the  glideslope  error  will  be 
recorded.  This  data  will  be  sorted  and 
analyzed  to  determine  how  many  feet  above 
glideslope  the  LSO  considers  to  be  'high',  and 
at  what  range  the  LSO  considers  to  be  the 
'middle'  of  the  approach.  Subsequently,  range 
and  glideslope  error  thresholds  can  be 
established  which,  if  exceeded,  will  cause  an 
automatic  'high  in  the  middle'  comment  to  be 
printed . Ranges  for  the  CAFQUAL  data  may  be 
slightly  different  from  those  collected  from 
the  NCLT.  This  could  occur  t ecause  NCLT  data 
is  being  collected  at  discrete  'windows', 
whereas  CARQUAL  data  is  being  collected  on  a 
continuous  analog  trace. 

For  example,  it  will  be  possible  to 
determine  the  range  from  touchdown  the  LSO 
considers  the  'middle'  of  the  pass.  (See  the 
vertical  dashed  line  in  Figure  6).  This  value 
will  be  determined  from  the  average  range  of 
the  flagged  data.  Glideslope  error  threshold 
values  (shown  by  the  horizontal  da6hed  line) 
will  also  be  determined.  This  will  be  accom- 
plished by  selecting  a value  for  the  glide- 
slope  error  threshold  that  nay  include,  for 
example,  75^  of  the  flagged  data. 

It  is  anticipated  that  a combination  of 
the  glideslope  error  thresholds  from  'high 
start',  'high  in  the  middle',  'high  in  close', 
and  'hig,h  at  the  ramp'  comments,  will  result 
in  more  objective  glideslope  scoring^hounda- 
ries.  Itiese  scoring  boundaries  will  be  com- 
pared to  the  results  of  the  regression  analy- 
ses. From  these  dual  analyses,  quantitative 
scoring  boundaries  will  be  identified. 


Figure  6.  Range  Versus  LSO  Comments 


TRAINING  AID  DEVELOPMENT 

One  of  the  products  of  this  program,  will 
be  an  automatic  NCLT  performance  trend  analy- 
sis printout  to  aid  the  LSO  in  the  NCLT  train- 
ing environment.  Development  of  this  training 
aid  will  be  accomplished  through  analysis  of 
LSO  suggestions  and  from  NCLT  and  CARQUAL 
data  collected  during  the  program.  LSC  com- 
ment data  will  be  examined  and  compared  with 
digital  and  analog  data  to  establish  boundary 
values  pertaining  to  specific  comments.  These 
data,  along  with  extensive  discussion  with 
LSO's,  will  be  used  to  develop  a performance 
trend  analysis  printout  addressing  those 
elements  which  the  LSO  considers  critical, 
dangerous,  or  unacceptable. 

The  Current  Training  Aid  Format 

The  training  aid  format  as  currently 
printed  out  in  the  NCLT  is  shown  in  Figure  7. 


The  specific  features  of  this  printout 

are : 

1.  Records  of  such  data  as  pilot's  name, 
5SN,  squadron,  date,  LSO's  name,  Landing  per- 
formance Score  (LPS),  wire  caught,  NCLT  lesson 
or  period,  number  of  runs  for  the  period,  and 
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the  total  number  of  runs  to  date. 

2.  Qualitative  comments  when  a para- 
meter boundary  is  exceeded  at  one  of  four 
windows  (3 A>  1/2,  l/U  NM  from,  touchdown,  and 
the  Ramp)  are  printed  ('left',  'right', 

'high',  'low',  'slow',  'fast',  etc.).  In 
addition,  the  performance  score  or  grade  for 
each  parameter  's  printed  under  the  heading 
"GRADF."  After  data  collection  and  analysis 
are  completed,  standards  for  each  parameter 
grade  will  be  determined  for  evaluating  an 
individual  pass,  i.e.,  how  an  individual's 
performance  in  lineup,  fer  instance,  compares 
to  t ie  population  or  standard  lineup  perfor- 
-ar.ee . 

3.  The  LSG  grade,  I.PS,  work  index, 
accuracy  index,  wire  caught  and  an  overall 
calculated  grade. 

h.  Appropriate  LSC  conner.ts  that  are 
entered  at  the  teletype.  Four  additional  dis- 
crete performance  items  or  comments  will  be 
automatically  flapped  for  use  in  the  printout. 
These  are:  "WING  ROCK  IN  CLOSE,"  "DROPPED 

NOSE  IN  CLOSE,"  "LANDED  ON  WAVEOFF,"  and  "APC 
IN  USE." 

A Refined  Version  of  the  Training  Format 

The  initial  NCLT  data  collection  software 
system  included  a preliminary  version  of  a 
performance  analysis  printout.  An  improved 
training  aid  will  be  developed  daring  the 
analysis  phase.  The  LSC’s  input  is  the  key 
element  in  this  development..  The  goal  of 
this  printout  is  to  assist  the  LSO  in  his 
training  task.  There  are  several  potential 
output  formats;  one  possibility  is  shown  in 
Figure  . 


Tills  printout  summarizes  an  entire  NCLT 
period.  Each  column  under  each  segment  of 
the  approach  corresponds  to  a separate  pass. 
In  the  example,  the  first  pass  was  high  on 
the  plideslope,  the  second  piss  was  on  the 
glidosiepe,  the  third  pass  was  low,  etc.  The 
wire  number  and  LSO  grade  are  summarized  in 
the  block  at  the  lower  right.  It  can  be  seen 
that  significant  trends  would  readily  show  up 


in  this  summary.  Coordination  with  the  LSO 

will  further  define  the  most  desirable 
summary  format. 

Current  NCLT  trend  analysis  comments 
flag  positional  and  range  parameters  that 
stray  from  nominal  values.  LSC’s  have  men- 
tioned that  the  decelerating  pass  never  gets 
identified,  nor  is  a good  correction  ever 
recognized  in  the  data  output.  Pilot  respon- 
siveness (acting  upon  or  ignoring)  to  a LSO 
command  is  also  not  considered.  Several  LSC ' 
and  pilots  have  mentioned  the  desirability 
of  giving  the  pilot  credit  for  exceptional 
performance.  The  LSO  will  do  this  occa- 
sionally with  comments  such  as  'good  correc- 
tion' or  'rails  pass'.  An  attempt  will  be 
made  to  explore  the  possibility  of  making 
similar  note  of  outstanding  performance  in 
the  NCLT.  It  is  felt  that  the  collection  of 
both  LSO  comment  data  and  objective  perfor- 
mance data  offers  a unique  opportunity  with 
which  to  explore  this  relatively  untouched 
region  of  performance  evaluation. 

For  example,  if  beyond  a certain  minir’in 
range,  a pilot  can  correct  an  excessive  (but 
not  dangerous)  positional  error  and  maintain 
the  optimum  flight  path  for  the  remainder  of 
the  pass,  such  a complimentary  comment  could 
be  made.  If  implemented,  the  feature  could 
prove  to  be  a strong  motivator  for  the  pilot 
in  the  simulator. 

In  addition  to  the  automatic  comments 
featured  in  the  current  training  aid  printout 
dangerous  performance  tendencies  or  'NO-NCs' 
will  be  investigated . In  con.i  mcl.iori  with 
this,  several  LSO's  have  mentioned  that  it 
may  be  desirable  to  call  greater  attertioi 
to  the  existence  of  such  unacceptable  per- 
formance. This  could  be  done  by  printing  a 
large  'X'  through  the  output.  In  order  to 
better  distinguish  performance  that,  is  dust 
outside  acceptable  boundaries,  LSO's  indicate 
'slightly  high',  'slightly  low',  'slightly 
fast',  etc.  The  LSO  currently  notes  this 
with  parentheses,  i.e.  (HIGH)  means  slightly 
high.  The  sane  scheme  will  be  considered  for 
the  automatic  trend  analysis  printout. 

Preliminary  Analysis  of  a Sample  of  Replace- 
ment. Pilots 

This  is  a quick  analysis  of  NCLT  data 
for  twenty-three  replacement  pilots . The 
sample  is  a mix  of  bb'r  who  subsequently 
reported  for  CAKQUAL  aboard  either  the  USS 
Saratoga  during  the  period  10-11  May  1°75, 
or  the  USS  Kitty  Hawk  during  the  period  2-C 
February  197f . 

Of  the  total  sa-ple  of  twenty-three 
id  's,  three  failed  to  qualify  during  night 
CARQ'JALS.  This  situation  offered  an  oppor- 
tunity to  test  for  the  predictive  capability 
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of  NCLT  performance  and  system  credibility. 

Two  other  pilots  failed  to  qualify  during 
these  CARQUALS.  However,  sufficient  NCLT 
data  for  these  two  pilots  were  not  available 
for  inclusion  in  this  analysis. 

Figure  9 compares  mean  NCLT  lineup  error 
data  for  the  three  HP's  who  failed  during 
CARQUAL  with  the  samples  of  20  RP's  who  did 
qualify.  For  reference,  values  for  the  total 
sample  are  shown  as  a dashed  line.  Percentage 
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Figure  9*  Mean  Lineup  Error  - Qualified 
Versus  Unqualified  Groups 

differences  between  those  who  qualified  and 
those  who  did  not  are  shown  at  the  midpoint 
of  each  segment.  The  differences  are  rather 
startling. 

Figure  10  is  a similar  comparison  for 
mean  glideslope  error. 


Figure  10.  Mean  Glideslope  Error  - 
Qualified  Versus  Unqualified  Groups 

These  figures  show  that  the  three  un- 
qualified individuals  performed  far  worse 
than  the  class  average  in  both  lineup  and 
glideslope  control.  The  percent  differences 


(A)  values  between  the  two  groups  are  shown 
at  five  points.  The  overall  A is  shown  in 
the  box  in  the  lower  center  of  the  figures . 

It  should  be  noted  that  the  data  plotted  re- 
flects arrested  landings  only.  Waveoff  and 
bolter  rates  for  the  three  unqualified 
individuals  were  also  significantly  higher 
than  the  class  average. 

Figure  11  compares  qualified  and  un- 
qualified group  performance  grades  for  lineup, 
glideslope,  fuel  flow,  angle  of  attack  and 
pitch  attitude.  This  comparison  shows  that 
the  unqualified  group  was  graded  lower  in 
lineup,  glideslope,  and  fuel  flow,  but  higher 
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Figure  11.  Parameter  Grades  - 

Qualified  Versus  Unqualified  Groups 

in  angle  of  attack  and  pitch  attitude.  Ttiis 
could  be  an  indication  that  the  unqualified 
group  tended  to  emphasize  control  of  these 
parameters  at  the  cost  of  control  in  the 
others  (i.e.,  cockpit  instrument  scan  versus 
position  cues).  The  lineup  and  fuel  flow 
deltas  appear  to  be  about  right,  but  the 
glideslope  grades  are  closer  than  the  prelimi- 
nary data  warrants.  TTiis  indicates  that  the 
glideslope  grading  equation  may  need  to  be 
more  reflective  of  "close  in"  error. 

Figure  12  compares  a student  who  was 
disqualified  during  NCLT  and  FCLP  with  another 
student  who  was  killed  as  a result  of  a ramp 
strike  and  with  the  two  samples  of  students 
who  either  qualified  or  were  disqualified  at 
the  ship.  It  should  be  noted  that  in  all 
cases  of  disqualification,  the  students  were 
graded  lower  in  lineup,  glideslope,  and  fuel 
flow  but  higher  in  angle  of  attack  and  pitch 
attitude.  'Biis  indicates  more  emphasis  by 
the  student  on  certain  parameters. 

The  foregoing  data  indicated  a strong 
predictive  relationship  between  NCLT  perfor- 
mance and  night  CARQUAL  performance,  even 
though  the  preliminary  analyses  to  date 
represent  only  rudimentary  efforts. 
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KiRure  1?.  Comparison  of  Computed  Parameter  Grades 
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1 RAINING  USING  INTERACTIVE  COMPUTER  GRAPHICS  FOR  SIMULATION 

ALICE  M.  CRAWFORD  and  RICHARD  E.  HURLOCK  , 

Navy  Personnel  Research  and  Development  Center 


Conventional  military  training  of  per- 
formance-oriented skills  generally  requires 
that  students  study  concepts  in  an  individ- 
ualized learning  module  and  then  practice  on 
actual  equipment  or  a high-fidelity  simu- 
lator. Not  only  are  these  training  programs 
costly,  but  considerable  research  has  shown 
that,  for  certain  skills,  such  hands-on 
practice  with  operational  equipment  may  not 
be  necessary  fo>"  adequate  transfer  of  train- 
ing (Hageman  and  Stockton,  1975). 

Rather  than  high-fidelity  appearance, 
the  critical  component  in  determining  the 
effectiveness  of  a training  device  may  be  the 
degree  to  which  students  are  allowed  to 
practice  behaviors  crucial  to  performance  in 
the  operational  situation  (Analysis  of  the 
Transfer  of  Training,  Substitution,  and 
Fidelity  of  Simulation  of  Training  Equip- 
ment, 1972).  Thus,  researchers  are  beginning 
to  focus  more  attention  on  the  underlying 
principles  of  learning  as  opposed  to  concen- 
trating on  creating  exact  replications  of  the 
actual  equipment. 

Consistent  with  this  trend,  the  present 
researchers  were  interested  in  developing 
and  evaluating  a training  methodology  which 
combined  low-fidelity  appearance  simulations 
with  established  principles  of  instruction. 2 
Specifically,  interactive  graphic  simulations 
presented  on  a computer-based  training  (CBT) 
instructional  system  were  investigated  with 
respect  to  their  potential  for  effective  low- 
cost  training. 

As  compared  to  conventional  methodolo- 
gies, interactive  computer  graphic  simulations 
could  offer  the  following  advantages: 

a.  Simulations  presented  on  a CBT 
system  may  be  combined  with  instruc- 
tional text  and  automatic  perfor- 
mance recordings  for  feedback  and 
later  evaluation. 

b.  A CBT  system  used  as  a training 
device  offers  a tremendous  amount  of 
flexibility  in  that  simulations  may 
be  programmed  within  the  framework 
of  any  learning  strategy  or  content 
sequencing  algorithm. 


c.  The  CBT  terminal  would  have  a 

general-purpose  simulation  capabil- 
ity in  that  the  various  operations 
on  different  pieces  of  equipment 
could  be  simulated  by  simply  access- 
ing different  lesson  software. 

Lahey,  Crawford,  and  Hurlock  (1975)  and 
Stern  (1975)  provided  initial  evidence  indi- 
cating the  potential  effectiveness  of  inter- 
active computer  graphic  simulations  for 
training.  In  both  studies  computer  graphics 
were  used  to  dynamically  simulate  equipment 
operations.  Students  were  able  to  interact 
with  these  simulations  as  they  would  with  the 
actual  equipment.  The  present  effort  was 
designed  to  be  a more  comprehensive  test  and 
evaluation  of  the  operational  and  technical 
feasibility  of  this  methodology.  More 
complex  simulations  were  programmed  and  more 
distinctly  performance-oriented  skills  were 
taught  than  in  the  previous  studies.  It  was 
felt  that,  if  effective  and  low-cost  train- 
ing was  demonstrated  in  this  study,  the  data 
would  serve  as  an  approximation  toward 
making  generalizations  for  using  similar 
applications  to  reduce  military  training 
costs. 

THE  COMPUTER-BASED  TRAINING 
PROGRAM 

PLATO  IV 

CBT  materials  were  programmed  and  run  on 
the  PLATO  IV  (for  Programmed  Logics  for 
Automated  Teaching  Operation)  Instructional 
System  which  is  driven  by  a CDC  Cyber  73 
computer.  Data  is  transmitted  between  the 
computer  and  remote  student  terminals  by 
voice  grade  telephone  lines.  Each  terminal 
had  a 64-character  keyboard,  an  8!j-inch 
square  plasma  display  panel,  and  touch  panels 
which  enabled  the  students  to  enter  data  into 
the  computer  when  they  pointed  at  any  of  the 
256  programmable  areas  of  the  plasma  panel. 
The  touch  panel  consists  of  a thin  framework 
which  is  substituted  for  the  normal  metal 
front  frame  of  the  PLATO  IV  terminal.  The 


^The  opinions  and  assertions  contained  herein  are  those  of  the  writers  and  are  not  to  be 
construed  as  official  or  as  reflecting  the  views  of  the  Navy  Department,  Naval  Service,  or 
Department  of  Defense. 
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This  project  was  jointly  sponsored  by  the  Joint  Services  Training  Technology  Program  of  the 
Defense  Advanced  Research  Projects  Agency. 
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panel  contains  the  necessary  electronics 
(scanning  system,  infrared  light  emitters 
and  detectors,  control  logic,  and  cable  to 
connect  with  the  terminal  connector)  to 
permit  the  panel  to  operate  as  an  auxiliary 
input  device.  The  light  emitters  and  detec- 
tors are  evenly  spaced  in  a 16  by  16  array. 

The  invisible  paths  formed  by  the  light  rays 
across  the  exterior  surface  of  the  plasma 
panel  describe  a checkerboard-like  array  of 
256Vinch  square  points  which  may  be 
selected  by  the  student.  Inputs  are  sensed 
by  the  system  when  the  beams  of  light 
emitting  from  the  X and  Y aces  are  broken 
simultaneously  by  the  student's  pointing 
responses.  Input  rate  is  limited  by  the 
touch  panel  circuitry  to  about  10  selections 
per  second.  Further  information  on  the 
system  is  provided  by  Stifle  (1972a,  1972b, 
1973). 

Training  Materials 

CBT  materials  developed  consisted  of 
interactive  computer  graphic  simulations  of 
the  front  panel  topography  and  all  operations 
of  the  S-3A  Copilot  Integrated  Control  System 
( I NCOS ) Panel  Multipurpose  Display  (MPD). 

Ihe  INCOS  panel  is  used  for  communications 
between  the  on-board  computer  and  the 
operator  for  the  utilization  of  nonacoustic 
sensors  and  computerized  navigation. 

Operator  interactions  with  the  panel  are 
reflected  in  changes  in  tactical  symbology 
and  tableaus,  readouts,  and  alerts  as  seen  on 
the  MPD.  Procedural  and  simple  perceptual 
motor  skills  are  required  to  operate  the 
INCOS  panel . 

The  computer-based  simulations  were 
accompanied  by  instructional  text,  guided 
practices,  extensive  feedback  and  test  se- 
quences designed  to  teach  the  copilot  the 
operations  of  each  panel  component.  All  mate- 
rials were  intended  to  elicit  maximum  par- 
ticipation on  the  part  of  the  student. 

The  PLATO  IV  touch  panel  was  used  to 
present  CBT  materials  in  a manner  which  would 
permit  students  to  respond  to  stimuli  from  the 
simulated  panel  simular  to  the  way  they  would 
in  the  operational  situation.  Displays  were 
created  by  interactive  computer  and  graphics 
in  which  the  students  could  touch  simulated 
panel  components  and  produce  responses  from 
the  panel  and/or  modifications  in  the  display 
on  the  MPD.  The  plasma  panel  displayed  clear, 
graphic,  and  close  to  actual  size,  simulations 
of  the  INCOS  panel,  and  was  able  to  produce 
all  of  the  same  visual  effects  which  are 
present  in  the  S-3A. 

The  training  materials  were  presented 
in  the  framework  of  a learner-controlled 


lesson  which  included  behavioral  objectives, 
examples,  guided  practices  and  diagnostic 
tests . 

A data  collection  system  was  programmed 
in  conjunction  with  the  training  materials 
to  record  the  following  data: 

a.  A trail  of  students'  progression 
through  the  lesson  and  each 
response  made. 

b.  Measures  of  time  spent  in  each 
practice  segment. 

c.  Correct  and  incorrect  responses 
made  to  each  test  item. 

Procedure 

Two  groups  of  S-3A  copilots  from  the 
Fleet  Avaition  Specialized  Operational  Train- 
ing Group  Pacific  Fleet  (FASOTRAGRUPAC)  were 
used  as  subjects  to  test  the  effectiveness  of 
the  computer-based  simulation  methodology. 

The  first  group  studied  INCOS  operations 
using  PLATO  IV  CBT  materials.  The  second 
underwent  conventional  FASOTRAGRUPAC  train- 
ing; that  is,  subjects  spent  8 hours,  study- 
ing a workbook,  which  described  each  of  the 
panel  components  in  conjunction  with 
sequential  pictures  of  panel  and  display 
actions  and  provided  test  questions.  Train- 
ing times  were  determined  by  time  required 
to  complete  the  materials.  Overall  training 
objectives  were  similar  for  both  forms  of 
training. 

Following  training,  both  groups  were 
administered  two  performance  posttests  — 
one  before  and  one  after  a 1 -hour  practice 
session  in  a high  fidelity  simulation 
position  trainer  (PT)  of  the  S-3A  copilot 
seat.  Repeated  measures  (test  performance 
before  and  after  the  practice  session)  were 
obtained  for  all  subjects.  During  this 
practice  session,  which  is  a standard 
scheduled  portion  of  FASOTRAGRUPAC  INCOS 
training,  the  instructor  guides  the  student 
tnrough  a series  of  exercises. 

Each  test  required  the  student  to 
perform  two  tasks  using  the  INCOS  panel. 

Each  task  was  in  the  form  of  a request  for  a 
terminal  behavior  and  required  from  6 to  10 
sequential  responses.  A maximum  of  3 min- 
utes was  allotted  for  successful  achievement 
of  each  task.  During  both  posttests,  an 
experimenter  sat  next  to  the  student  to 
maintain  a checklist  of  the  sequence  of 
student  responses  made  on  each  task  and  to 
record  the  time  required. 
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Results 

The  dependent  measures  used  for  data 
analysis  were  the  number  of  tasks  completed 
(0,  1 , or  2)  in  each  test,  and  the  time 
required  to  complete  ea'rh  test  (the  maximum 
possible  was  6 minutes). 

Two-way  (groups  by  form  of  test  admin- 
istered) analyses  of  variance  were  performed 
on  all  data  to  compare  the  performance  of: 

(1)  CBT  students  versus  workbook  students  on 
Test  1,  (2)  CBT  students  versus  workbook 
students  on  Test  2,  and  (3)  CBT  students  on 
Test  1 versus  that  of  workbook  students  on 
Test  2.  Table  1 presents  the  mean  score  for 
both  groups  on  the  two  dependent  measures. 

On  the  performance  test  given  imme- 
diately after  training,  analysis  of  the  data 
revealed  that  CBT  students  completed  more 
Problems  (F  = 9.92,  df  = 1/40,  p < .005),  and 
finished  more  quickly  (F  - 23.29,  df  = 1/40, 
p <.001)  than  did  workbook  students. 

After  a 1-hour  practice  session  in  the 
PT,  CBT  students  were  still  performing  better 
than  workbook  students.  Their  performance 
scores  for  Test  2 showed  that,  again,  they 
completed  more  problems  (F  = 5.29,  df  = 1/40, 
p < .05),  and  their  tests  were  completed  in 
less  time  (F  * 16.68,  df  = 1/40,  p < .001). 


A final  analysis  was  performed  on  data 
obtained  from  CBT  students  on  Test  1 and 
workbook  students  on  Test  2.  This  comparison 
revealed  no  significant  differences  between 
scores  of  the  workbook  students  who  had  had 
hands-on  experience  in  the  PT  and  those  of 
the  CBT  students  who  had  not  yet  practiced 
with  the  actual  equipment. 

Data  were  also  collected  from  the  in- 
structors at  FASOTRAGRUPAC.  Given  that  these 
men  have  had  extensive  experience  with  the 
INCOS  system,  it  was  felt  that  their  scores 
would  reflect  a realistic  estimate  of  the 
minimum  time  for  the  performance  on  both  forms 
of  the  test.  Instructor  and  student  time  data 
is  shown  in  Table  2. 

An  incidental  finding  was  that  after  3 
hours  of  training  on  PLATO  IV,  and  following 
the  performance  tests  in  the  PT,  68%  of  the 
experimental  students  returned  to  study  the 
PLATO  INCOS  lesson  (on  their  own  time)  for 
1.5  to  3.7  additional  hours.  Average  total 
time  on  the  system  for  all  CBT  students  was 
4.8  hours. 


TABLE  1.  COMPARISON  OF  GROUP  PERFORMANCE  ON  TESTS  1 AND  2 


Test  1 Test  2 

Time  No.  Tasks  Time  No.  Tasks 
Group  (Min.)  Completed  (Min.)  Completed 


Computer-Based  Training  2.85 
Workbook  4.44 


1.68  1.88  2.00 

1.18  3.32  1.73 


TABLE  2.  MEAN  TIMES  TO  COMPLETE  PERFORMANCE  TESTS  FOR  INSTRUCTORS  AND  TRAINEES 

(In  Minutes) 


Computer-Based 

Workbook 

Instructors 

Training  Group 

Group 

Test 

Mean 

SD 

Mean 

SD 

Mean 

SD 



.63 

.14 

2.85 

1.20 

4.44  T. 30 

2 

.67 

.21 

1.87 

.84 

3.32 

1.49 
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EVALUATION 

Training  Effectiveness 

Results  of  analysis  of  the  data 
collected  from  both  dependent  measures 
clearly  indicated  that  the  CBT  with  inter- 
active graphics  provided  training  which  was 
superior  to  that  provided  by  the  conventional 
program.  The  findings  suggest  that  CBT 
materials  can  be  used  operational ly  either  to 
supplement  or  replace  conventional  training 
programs. 

The  present  data  support  the  possibility 
that  supplementation  of  a conventional  train- 
ing program  with  computer-based  simulations 
might  permit  students  to  obtain  maximum 
benefits  from  experience  with  the  operational 
equipment.  While  both  control  and 
experimental  groups  showed  improved  perform- 
ance following  the  PT  exercises,  CBT 
students  performed  significantly  better  than 
the  control  group.  Given  that  hands-on 
experience  with  the  operational  equipment  did 
not  bring  all  students  to  the  same-level  of 
mastery,  as  indicated  by  instructor  data, 
it  is  reasonable  to  conclude  that  CBT  plus 
hands-on  experience  with  actual  equipment 
provided  the  best  training. 

While  supplementing  a conventional  pro- 
gram with  a computer-based  simulation  of 
operational  equipment  may  be  beneficial  in 
some  situations,  eliminating  the  conventional 
program  in  favor  of  the  computer-based 
simulation  may,  under  some  circumstances, 
provide  the  most  efficient  training.  This 
could  prove  to  be  the  case  with  the  S-3A 
INCOS  training  program.  An  important  finding 
of  the  research  was  that  CBT  students  per- 
formed required  behaviors  in  the  PT  as  well 
before  the  practice  exercises  as  convention- 
ally trained  students  did  after  the  exercises. 
Because  the  S-3A  staff  has  always  considered 
conventional  student  proficiency  level  follow- 
ing PT  exercises  acceptable,  the  findings  of 
this  research  imply  that  CBT  could  be  used 
to  replace  the  PT  during  this  phase  of 
copilot  training. 

The  fact  that  performance  was  not  de- 
graded by  the  absence  of  conventional  hands- 
on  practice  has  important  implications  for 
training.  It  appears  that  computer-based 
graphic  simulations  may  be  an  appropriate 
methodology  for  teaching  various  levels  of 
performance-oriented  skills.  Precise 
specification  of  these  levels  will  be  in- 
vestigated by  further  research.  It  is  not 
clear,  however,  whether  a less  expensive 
media,  such  as  slides,  would  have  produced 
equally  effective  training.  It  does  seem 
possible  that  higher  level  skills  may  be 


effectively  taught  using  computer-based 
simulations. 

The  materials  developed  for  the  INCOS 
simulation  may  be  easily  converted  for  use 
in  other  phases  of  copilot  training  or  for 
training  other  S-3A  operators.  Thus,  it  is 
advantageous  to  attempt  to  isolate  the 
specific  variables  responsible  for  the 
success  with  this  methodology. 

Cost  Benefits 

Costs  associated  with  delivering  computer- 
based  training  with  a PLATO  IV  system  are 
a function  of  the  number  of  hours  the  term- 
inals are  used  per  year,  and  whether  they  are 
purchased  or  leased.  Figuring  by  the  least 
expensive  way,  that  is,  assuming  that 
terminals  are  purchased  with  their  cost 
amortized  over  10  years  and  they  receive 
maximum  utilization,  the  cost  per  student 
contact  hour  is  about  $5.00,  including 
time-sharing  expenses.  Thus,  since  each 
student  in  this  study  had  3 hours  of  PLATO 
training,  CBT  was  delivered  at  a cost  of 
$15.00  per  student. 

The  cost  of  the  conventional  training 
this  could  replace  may  be  conservatively 
estimated  at  $118.00.  This  figure  is  based 
on  amortization  of  the  costs  of  the  position 
trainer  and  its  maximum  utilization. 

The  implications  of  the  cost  figures  are 
that  $20,000.00  per  year,  or  almost  a quarter 
million  dollars  over  a 10-year  period,  could 
be  saved.  Additional  savings  include  5 stu- 
dent hours  and  1 instructor  hour  for  each 
man  trained,  and  the  time  saved  on  the  PT 
which  may  then  be  used  in  more  advanced 
training  situations. 

CONCLUSIONS 

The  present  study  has  demonstrated  tne 
usefulness  of  CBT  with  interactive  graphic 
simulations  for  part-task  training  of  certain 
performance-oriented  skills.  Using  this 
methodology  to  supplement  or  replace  some 
conventional  training  programs  which  use 
actual  equipment  or  high  fidelity  simulators 
for  hands-on  training  appears  to  be  tech- 
nically and  operationally  feasible.  Graphic 
simulations  have  numerous  capabilities  which 
are  advantageous  for  training  as  compared  to 
the  conventional  programs.  C8T  systems  can 
combine  simulations  of  operational  equip- 
ment with  any  instructional  strategy,  feed- 
back, recording  of  student  performance  data, 
and  instructional  sequencing. 

The  PLATO  IV  instructional  system  was 
demonstrated  to  be  a good  vehicle  for  pre- 
senting interactive  training  materials. 
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Cost  figures  show  that,  with  full  terminal 
utilization,  PLATO  IV  can  provide  cost- 
effective  training  as  compared  to  the  con- 
ventional training.  The  system  also  offers 
considerable  potential  as  a general-purpose 
training  device.  Different  pieces  of  equip- 
ment may  be  simultaneously  simulated  at 
different  terminals  or  a piece  of  simulated 
equipment  may  be  updated  by  simply  accessing 
different  lesson  software. 
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MAXIMIZING  FLIGHT  FIDELITY; 

INTEGRATION  OF  NAVAL  AIR  TEST  CENTER  CAPABILITIES 
INTO  THE  PROCUREMENT  OF  MAJOR  AVIATION  TRAINING  DEVICES 

R.  T.  GALLOWAY 
Naval  Air  Test  Center 


Flight  test  engineers  and  test 
pilots  from  the  Naval  Air  Test  Center 
(NATC)  have  recently  become  involved 
with  evaluating  the  flight  fidelity  of 
new  Operational  Flight  Trainers  (OFT) 
and  Weapon  Systems  Trainers  (WST)  be- 
ing procured  by  the  U.  S.  Navy  and 
Marine  Corps.  The  purpose  of  all  NATC 
flight  fidelity  evaluations  is  to 
answer  the  question:  Does  this  Device 

fly  like  the  airplane?  To  formulate 
such  an  answer,  these  engineers  and 
test  pilots  compare  OFT/WST  simulated 
flying  qualities,  performance,  engine 
systems,  and  weapons /avionics  systems 
characteristics  to  those  of  the  actual 
airplane.  The  major  component  of  NATC 
testing  is  the  application  of  quanti- 
tative test  techniques  to  identify 
fidelity  deficiencies  and  eliminate  de- 
pendence on  purely  subjective  analysis. 
The  quantitative  test  techniques  ap- 
plied here  are  generally  identical  to 
those  routinely  utilized  by  NATC  during 
aircraft  flight  test  programs.  The 
extent  of  NATC  involvement  in  OFT/WST 
testing  is  outlined  in  table  1. 

NATC  Test  Background 

The  type  of  airplane  testing  con- 
ducted by  NATC  is  definitely  germane 
to  simulator  fidelity  testing.  In 
brief,  the  mission  areas  of  NATC  that 
are  relevant  to  OFT/WST  fidelity  in- 
clude testing  of  aircraft  flying  quali- 
ties, performance,  carrier  suitability, 
propulsion  systems,  and  mission  sys- 
tems (avionics,  F.CM,  radar,  reconnais- 
sance sensors)  for  all  Navy  and  Marine 
Corps  fixed  wing,  rotary  wing,  and 
VSTOL  aircraft.  The  normal  NATC  cycle 
begins  with  Navy  Preliminary  Evalua- 
tions (NPE)  conducted  at  the  contrac- 
tor's facility  at  various  stages  of 
the  aircraft  and  system  development. 

The  purpose  of  these  NPE's  is  to  dis- 
cover serious  deficiencies  early 
enough  to  allow  correction  by  the  con- 
tractor without  delay  in  delivery  to 
the  fleet.  The  guidelines  for  such 
testing  are  contained  in  the  demon- 
stration specification,  MIL-D-8708, 
which  calls  for  five  NPE  phases  with 
the  first  phase  occurring  within  90 
days  after  the  first  flight  of  the  new 
aircraft.  After  the  NPE  phases  of 


testing  are  complete,  fleet  repre- 
sentative aircraft  are  evaluated  at 
NATC  with  contractor  demonstrations 
of  contract  guarantee  items  and  Navy 
Board  of  Inspection  and  Survey  (BIS) 
service  acceptance  trials.  Finally, 
a follow-on  technical  evaluation  is 
usually  conducted  at  NATC  to  document 
pertinent  characteristics.  Such  a 
test  process  usually  results  in  large 
quantities  of  data  at  NATC  concerning 
airplane  and  system  characteristics. 
Unfortunately,  time  and  money  con- 
straints generally  limit  published 
data  to  documentation  of  deficiencies 
and  flight  characteristics  at  the 
edges  of  the  flight  envelope.  As  a 
result,  the  mid-envelope  character- 
istics needed  to  assess  OFT/WST 
flight  fidelity  remain  undocumented 
and  further  flight  tests  are  required. 
Data  availability  and  flight  test  re- 
quirements are  discussed  in  depth  in 
later  paragraphs. 

OFT/WST  Test  Policy 

The  initial  role  of  NATC  in 
OFT/WST  testing  was  only  to  provide 
airplane  data  in  areas  otherwise  un- 
available to  the  simulator  contrac- 
tor. It  soon  became  apparent  that 
NATC  airplane  test  techniques  were 
as  useful  for  quantifying  OFT/WST 
deficiencies  as  they  were  in  the  air- 
plane. Unfortunately,  these  initial 
efforts  by  NATC  occurred  during  final 
acceptance  phases  of  each  OFT/WST  and 
considerable  delays  were  incurred 
since  extensive  software  changes  were 
usually  required  by  the  contractor  to 
correct  the  flight  fidelity  deficien- 
cies. For  instance,  in-plant  ac- 
ceptance of  the  T-2C  OFT  (Device 
2F101)  was  delayed  by  approximately 
six  months  primarily  for  flight  fi- 
delity problems  and  the  A-4M  OFT 
(Device  2F108)  in-plant  acceptance 
was  similarly  delayed  by  four  months. 
In  order  to  eliminate  these  costly 
delays,  NATC  simulator  project  per- 
sonnel conferred  with  representatives 
of  the  Naval  Training  Equipment 
Center  (NTEC)  and  the  Naval  Air 
Systems  Command  (NAVAIR)  to  develop 
the  most  effective  means  of  incor- 
porating NATC  into  the  procurement 
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TABLE  1.  NATO  OFT/WST  TEST  EXPERIENCE 


Device 

Test  Scope 

Test  Period 

T-2C  OFT  (2F101) 

Acceptance  testing  (Units  1-7) 

1973-1975 

TA-4J  OFT  (2F90) 

Fidelity  improvement  for  use 
with  visual  system 

1974-1975 

S-3A  WST  (2F92) 

a.  Acceptance  testing  (Units 
1 and  2) 

b.  VITAL  III  retrofit  and  WST 
aero  update  (Unit  2) 

c.  Acceptance  testing  (Units 
3 - 5) 

1974- 1975 

1975- 1976 

1976- 1977 

F-14A  OFT  (2F95) 

Aero  updates  plus  retrofit  of 
VITAL  II  visual  evaluation 
(Units  2.  3,  4) 

1974-1976 

A-7E  NCLT  (2F103) 

a.  Fidelity  evaluation 

b.  Aero  update  evaluation 

1974 

1976 

A-7E  WST  (2F111) 

a.  NPE  (Unit  1) 

b.  Acceptance  testing  (Units 
l and  2) 

1975 

1975-1976 

A-4M  OFT  (2F108) 

Acceptance  testing  (Units 
1 and  2) 

1975-1976 

KC-130F  OFT  (2F107) 

Acceptance  testing 

1975-1976 

SH-2F  WST  (2F106) 

a.  Acceptance  testing  (Units 
1 and  2) 

b.  VITAL  III  visual  retrofit 

1975-1976 

1976 

F-4J  WST  (2F88) 

Acceptance  testing  (USMC) 

1976 

EA-6B  WST  (2F119) 

a.  Proposal  evaluation 

b.  Monitor  development,  base- 
line data  generation 

c.  Testing  (NPE.  Acceptance) 

1975 

1976 

1977-1978 

A-6E  WST  (2F114 ) 

a.  Detail  specification  comment 

b.  Baseline  data  generation 

c.  Acceptance  testing  (Units 
1 and  2) 

1974 

1976- 1977 

1977- 1978 

FUTURE  PROGRAMS: 

E-2C  OFT  (2F110) 
F-14A  WST  (2F112) 
CH-47E  OFT  (2F117) 
ACMT  (2E6) 

a.  Monitor  development 

b.  Testing  (NPE,  Acceptance) 

1976- 

1977- 
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cycle  of  new  OFT's  and  WST's  so  that 
flight  fidelity  would  be  assured  and 
realistic  development  and  testing 
schedules  could  be  written.  The  major 
elements  of  the  Navy  test  policy  so  de- 
veloped are  outlined  below.  It  should 
be  noted  that  the  NATO  OFT/WST  project 
engineer /test  pilot  team  participates 
only  in  as  many  of  the  tasks  as  re- 
quired bv  NAVAIR.  The  major  elements 
are : 

a Kstablish  liaison  with  appro- 
priate NAVAIR  and  NTEC  personnel. 

b.  Assist  in  specification  prepa- 
ration, proposal  evaluation,  and  moni- 
tor the  contractor's  development  effort, 
where  possible,  to  assure  flight  fidel- 
ity. 

c.  Provide  NATC -developed  flight 
test  data  as  needed  for  OFT/WST  de- 
velopment and  as  standard  for  measure- 
ment of  flight  fidelity. 

d.  Conduct  Navy  Preliminary  Evalu- 
ations (NPE's)  of  the  OFT/WST  during 
development . 

e.  Participate  in  in-plant  ac- 
ceptance testing. 

f.  Provide  engineering  and  test 
pilot  assistance  in  adjusting  hard- 
ware and  software  to  achieve  proper 
flight  fidelity. 

g.  Participate  in  on-site  accep- 
tance to  validate  OFT/WST  flight  fi- 
delity . 

h.  Participate  in  validation  of 
subsequent  OFT/WST  units  during  in- 
plant  and  on-site  acceptance. 

i.  Conduct  follow-on  evaluations 
of  update  (ECP)  efforts  affecting 
flight  fidelity. 

j.  Prepare  reports  documenting 
flight  fidelity  of  the  OFT/WST. 

The  major  innovations  to  the  OFT/ 
WST  procurement  process  are  items  c. 
and  d.  from  the  above  list.  The  ad- 
vantage of  incorporating  NATC -developed 
flight  test  data  as  early  as  possible 
to  describe  the  characteristics  to  ' e 
simulated  is  obvious.  The  introduction 
of  the  NPE  concept  to  simulator  flight 
fidelity  testing  is  a direct  adaptation 
of  U.  S.  Navy  aircraft  procurement 
policy.  The  advantage  of  early  de- 
ficiency detection  through  brief  evalu- 
ations of  the  OFT/WST  during  the 
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development  phase  is  that  more  time 
is  available  to  the  contractor  to 
develop  improvements  by  obtaining 
more  data  and  to  refine  the  hardware 
and  software.  To  date,  the  only 
simulator  we  have  been  able  to  evalu- 
ate under  the  NPE  concept  was  the 
A-7E  WST,  Device  2F111,  in  January 
1975,  where  NATC  conducted  a four- 
day  period  of  tests  of  flying  quali- 
ties and  performance.  This  evalu- 
ation was  possible  only  because  of  a 
gap  in  the  development  process  (the 
radar  simulation  equipment  was  not 
ready)  but  it  did  reveal  some  prob- 
lem areas  in  time  to  obtain  appro- 
priate data  for  clarification  of 
these  problems  before  acceptance 
Jesting  began.  For  the  future, 
similar  NPE  efforts  are  planned  for 
the  E-2C  OFT  (Device  2F110)  and 
EA-6B  WST  (Device  2F119) . The  number 
of  NPE's  required  for  any  particular 
OFT/WST  will  vary  with  the  complexity 
of  the  simulator  (OFT  vs.  WST)  and 
the  development  progress  (readiness 
of  avionics,  motion  system,  etc.). 
Regardless  of  the  number  of  NPE's, 
each  NPE  is  intended  to  be  a three 
to  five-day  evaluation  on  a frozen- 
configuration  at  significant  mile- 
stones during  the  development  phase. 

OFT/WST  TEST  EXPERIENCE 

Baseline  Data  Accumulation 

Regardless  of  what  point  we 
enter  the  OFT/WST  development  cycle, 
the  NATC  project  team  always  begins 
baseline  data  accumulation  by  search- 
ing out  appropriate  published  and  un- 
published flight  test  data.  If  the 
airplane  being  simulated  is  still 
being  tested  in  the  normal  NATC  test 
cycle,  then  usually  a wealth  of  use- 
ful but  unpublished  data  is  avail- 
able. This  was  the  case  during  ac- 
ceptance testing  of  the  S-3A  WST  and 
and  the  F-14A  OFT.  However,  the  ap- 
proach of  " piggy-backing  " OFT/WST 
test  requirements  with  airplane  de- 
velopment testing  produces  only 
limited  results  since  data  require- 
ments may  differ  and  airplane  program 
requirements  take  priority  over  simu- 
lator requirements  in  competing  for 
expensive  flight  time  in  a fully 
instrumented  airplane. 

For  most  of  our  OFT/WST  programs, 
insufficient  data  exist  so  that  ad- 
ditional flight  tests  are  required  to 
build  the  required  data  base,  and 
this  is  generally  done  with  uninstru- 
mented fleet  airplanes.  However, 


data  gathering  in  fleet  aircraft  holds 
several  advantages  in  that  the  fleet 
airplane  is  usually  more  representa- 
tive of  the  simulator  configuration 
than  the  test  articles  at  NATC . In 
addition,  testing  of  several  fleet 
airplanes  allows  documentation  of 
average  airplane  characteristics,  par- 
ticularly with  respect  to  control  sys- 
tem characteristics.  Normally,  about 
15  hours  of  flight  time  are  required 
to  build  an  adequate  data  base,  and 
tests  conducted  include  documentation 
of  mid-envelope  airframe  and  engine 
characteristics,  response  to  small 
inputs  in  all  axes,  and  open  loop 
characteristics  at  high  angles  of  at- 
tack or  during  configuration  changes. 
Use  of  relatively  simple  instrumenta- 
tion such  as  hand-held  force  gauges, 
stop  watches,  tape  measures,  and  pro- 
duction cockpit  instruments,  has  proven 
to  be  adequate  and  allows  considerable 
flexibility  in  going  from  one  cockpit 
to  another,  be  it  airplane  or  simula- 
tor. The  use  of  such  austere  instru- 
mentation also  permits  rapid  data 
generation  in  the  airplane  should  sig- 
nificant gaps  or  questionable  areas 
arise  during  simulator  testing.  Also, 
the  use  of  identical  instrumentation 
in  the  airplane  and  simulator  enhance 
the  repeatability  of  test  techniques 
and  data. 

We  do  not,  however,  consider  the 
extensive  use  of  hand-held  instrumen- 
tation a panacea.  If  additional  mean- 
ingful test  data  from  an  instrumented 
airplane  becomes  available  during  a 
simulator  test  period,  we  do  not  hesi- 
tate to  use  it.  Of  course,  it  would 
be  far  more  desirable  if  such  data 
were  available  in  advance  so  that  a 
contractor  could  detect  a fidelity 
problem  before  Navy  testing  commences. 
The  effort  is  underway  within  the  Navy 
to  apprise  aircraft  program  managers 
of  simulator  data  requirements  so  that 
funding  is  available  to  publish  all 
pertinent  data  and  so  that  aircraft 
test  programs  can  include  maneuvers 
for  simulator  needs  along  with  the 
usual  airplane  specification  test  re- 
quirements. Hopefully,  this  effort 
will  lead  to  more  complete  documenta- 
tion of  airplane  characteristics  in 
time  to  aid  a simulator  contractor 
during  his  development  phase. 

Simulator  Testing 

Flight  fidelity  tests  on  an  OFT/ 
WST  are  conducted  merely  by  repeating 
airplane  test  techniques  at  the  base- 
line data  test  conditions.  It  is  not 


desirable  for  the  NATC  tests  to  ad- 
here to  some  prearranged  detailed  out- 
line of  tests,  such  as  the  Acceptance 
Test  Procedures  Report,  since  the  test 
techniques  utilized  by  a test  pilot 
cannot  always  be  described  adequately 
in  a step-by-step  manner  to  enable  a 
layman  to  obtain  repeatable  results. 

It  is  preferable  that  flying  quali- 
ties tests  such  as  required  to  assess 
maneuvering  longitudinal  stability  be 
left  to  a trained  test  pilot.  In  ad- 
dition, fidelity  problems  may  require 
that  a majority  of  the  NATC  testing 
be  concentrated  in  one  area,  such  as 
lateral  control  sensitivity,  where 
additional  airplane  flight  tests  might 
be  required  for  data  and  the  test  con- 
ditions would  not  be  defined  to  an 
exact  gross  weight  or  airspeed  until 
that  time.  For  this  reason,  a general 
outline  of  the  tests  to  be  conducted 
and  the  flight  envelope  to  be  inves- 
tigated is  considered  the  most  mean- 
ingful planning  document  that  NATC 
can  provide  a simulator  contractor  in 
anticipation  of  our  flight  fidelity 
testing,  and  we  have  provided  such  an 
outline  prior  to  testing  when  re- 
quested . 

The  bulk  of  NATC  simulator  test- 
ing is  conducted  on  the  fixed-base 
cockpit  without  motion  or  visual  cues 
since  the  primary  flight  fidelity  goal 
is  to  have  the  airplane  response  as 
seen  on  the  simulator  cockpit  instru- 
ments and  controls  match  the  response 
as  seen  on  the  airplane  cockpit  in- 
struments and  controls.  Obviously, 
the  OFT/WST  flight  dynamics  (hardware 
and  software)  must  match  on  this  basis 
before  a meaningful  evaluation  of  the 
motion  and  visual  cues  can  be  con- 
ducted . 

Another  facet  of  flight  fidelity 
testing  is  pilot  "memory. " During 
our  evaluations,  we  have  found  that 
it  doesn't  take  a test  pilot  in  the 
simulator  very  long  to  forget  subtle 
airplane  characteristics.  For  in- 
stance, during  tests  of  the  S-3A  WST 
with  the  VITAL  III  visual  system,  an 
experienced  NATC  S-3A  test  pilot  be- 
came adapted  to  the  simulation  within 
only  30  minutes;  that  is,  during  his 
first  30  minutes  of  flying  landing  ap- 
proaches in  the  WST,  he  used  normal 
airplane  pilot  technique  and  he  en- 
countered extreme  difficulty  in  making 
smooth  approaches.  Thereafter,  he 
learned  to  compensate  for  the  fidelity 
problems,  his  approaches  became 
smooth,  and  his  qualitative  analyses 
of  the  simulation  lost  considerable 
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value.  The  only  way  to  alleviate  such 
a problem  is  for  the  pilot  to  periodi- 
cally fly  the  airplane  during  the  OFT/ 
WST  test  program  especially  since  there 
are  still  many  subtle  flight  character- 
istics, such  as  closed-loop  flying 
qualities  in  the  approach,  which  elude 
quantification  and  can  only  be  defined 
properly  by  refreshed  pilot  opinion. 
This  practice  of  pilot  refamiliariza- 
tion was  utilized  by  the  NATO  test 
teams  particularly  during  the  extensive 
evaluations  conducted  on  the  T-2C  OFT 
and  the  TA-4J  OFT  where  considerable 
flight  time  was  graciously  provided  by 
the  Naval  Air  Training  Command. 

Deficiency  Resolution 

When  the  NATC  test  team  encounters 
a flight  fidelity  deficiency  during  ac- 
ceptance testing,  the  problem  is  des- 
cribed with  as  much  data  as  possible  in 
the  deficiency  report.  The  intention 
is  to  describe  the  problem  well  enough 
to  permit  the  contractor  to  develop 
corrections  without  the  presence  of  the 
NATC  test  team.  The  workload  of  the 
NATC  test  pilots  and  engineers  always 
includes  other  projects  with  varying 
priority  requirements  so  that  extensive 
tweaking  periods  at  the  simulator  are 
a luxury  in  which  we  cannot  indulge. 
Therefore,  the  NATC  OFT/WST  test  team 
endeavors  to  assist  the  contractor  as 
much  as  possible  in  isolating  a flight 
fidelity  problem,  but  it  must  be  recog- 
nized that  presence  of  the  NATC  test 
team  at  the  simulator  is  of  necessity 
limited  to  relatively  short  intervals. 

Typical  Fidelity  Problem  Areas 

The  type  of  testing  provided  by 
NATC  can  be  illustrated  by  discussing 
typical  fidelity  problem  areas  en- 
countered in  our  OFT/WST  testing.  One 
of  the  most  critical  problem  areas  we 
have  encountered  is  in  control  loading 
fidelity  where  control  stick  breakout 
forces,  damping,  and  centering  charac- 
teristics are  found  to  be  unrepresenta- 
tive of  the  airplane.  Many  times  this 
is  due  to  the  use  of  control  system 
design  data  vice  actual  airplane  test 
data,  and  once  the  correct  data  are 
made  available,  only  simple  adjustments 
are  necessary.  However,  some  control 
loading  designs,  particularly  the 
electo-hydraulic  type,  seem  to  require 
extensive  care  to  retain  alignment 
which  is  undesirable  from  a maintenance 
standpoint.  Obviously,  poor  control 
loading  fidelity  can  seriously  degrade 
an  otherwise  excellent  flight  dynamics 
software  model. 


The  flight  dynamics  software 
model  suffers  in  fidelity  from  de- 
pendence on  wind  tunnel  data,  as 
everyone  knows.  Therefore,  the  NATC 
test  team  devotes  considerable  atten- 
tion to  areas  such  as  the  high  angle 
of  attack  regime  since  baseline  data 
are  usually  sketchy  but  NATC  test 
pilots  have  extensive  flight  test  ex- 
perience in  this  regime.  In  most 
cases,  a representative  simulation  of 
airplane  stall  characteristics,  in- 
cluding motion  base  buffet  cues,  can 
be  developed  by  an  experienced  test 
pilot  working  in  conjunction  with  con- 
tractor software  experts.  In  addition, 
many  flight  fidelity  problems  are  com- 
pounded by  software  short  cuts  such  as 
low  computation  rates  to  save  computer 
time  and  delayed  output  of  computed 
variables  such  as  roll  angle  to  the 
pilot's  gyro.  Admittedly,  software 
architecture  is  out  of  the  NATC  realm 
of  expertise  but  these  problems  are 
mentioned  only  because  we  can  recog- 
nize their  effects  on  fidelity  when 
trying  to  assist  the  contractoi  in 
identifying  the  source  of  a flight 
fidelity  problem.  Finally,  the  re- 
finement of  visual  scene  response 
has  always  proven  to  be  difficult, 
especially  for  closed-loop  tasks  such 
as  landing  approaches.  Some  success 
has  been  achieved,  particularly  with 
the  TA-4J  OFT  at  NAS  Kingsville  and 
the  S-3A  WST/VITAL  III  at  NAS  Cecil 
Field,  by  careful  adjustment  of  ap- 
propriate software  terms  while  re- 
peating a well  defined  task  until  a 
representative  simulation  is  achieved. 

Simulator  Certification 

As  the  dependence  on  OFT's  and 
WST's  increases  in  maintaining  pilot 
proficiency  in  the  U.  S.  Navy,  so 
will  the  demand  that  adequate  fidelity 
be  developed  and  retained  in  these 
simulators.  Any  future  simulator  cer- 
tification programs  should  definitely 
include  flight  fidelity  evaluations  by 
NATC  test  pilots  and  engineers.  Such 
evaluations  would  be  similar  to  the 
brief  on-site  validation  tests  cur- 
rently conducted  when  a new  device  is 
first  installed,  but  they  would  serve 
to  confirm  adequate  fidelity  or  help 
define  any  new  problem  areas  which 
might  have  crept  in  due  to  control 
loading  degradation  or  inadvertent 
software  changes.  Our  experience  with 
the  TA-4J  OFT  and  the  S-3A  WST  showed 
that  these  problems  do  occur.  There- 
fore, the  use  of  quantitative  testing 
by  an  NATC  project  team,  particularly 
in  flying  qualities  evaluations,  is 
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just  as  necessary  for  certification  as 
in  acceptance  testing. 

CONCLUS ION 

In  summary,  the  utilization  of 
NATO  flight  test  data  and  quantitative 
test  techniques  in  OFT/WST  flight  fi- 
delity evaluations  has  increased  stead- 
ily since  1973.  An  orderly  process  for 
including  NATO  inputs  into  the  OFT/WST 
procurement  process  as  early  as  possible 
has  been  developed  in  conjunction  with 
the  Naval  Air  Systems  Command  and  the 
Naval  Training  Equipment  Center.  An 
effort  is  underway  to  include  simulator 
data  requirements  with  existing  air- 
craft testing  requirements  for  future 


airplane  and  OFT/WST  procurements. 
Current  NATC  flight  fidelity  test  pro- 
grams have  shown  that  meaningful  data 
can  be  obtained  with  simple  hand-held 
instrumentation,  that  the  test  pilot 
must  frequently  refamiliarize  himself 
with  subtle  airplane  characteristics 
during  the  simulator  evaluation  period, 
and  that  NATC  evaluation  periods  are 
best  conducted  over  brief  periods  be- 
ginning sometime  in  the  development 
phase.  Hopefully,  such  extensive  ef- 
forts to  achieve  maximum  flight  fi- 
delity will  increase  the  training 
value  of  OFT's  and  WST's,  or  at  least 
make  the  use  of  such  devices  more 
palatable  to  Naval  aviators  than  in 
the  past. 
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PERFORMANCE  ORIENTED  AIRCREW  TRAINING: 
OPTIMIZATION  THROUGH  ISP* 


W.  M.  HINTON,  JR.  and  R.  P.  FISHBURNE,  JR. 
Cal  span  Corporation 


Traditionally,  training  programs  within 
the  military  have  been  generated  in-house  by 
a number  of  instructor  personnel  tasked  with 
the  difficult  problem  of  sorting  through  all 
available  information  and  developing  an 
approach  to  the  classroom  presentation  of  an 
assigned  content  area.  The  resulting  in- 
structor guides  (outlines)  and  student  hand- 
outs are,  by  necessity,  based  on  information 
within  each  individual's  realm  of  experience. 
This  experience  is  largely  based  on  prior 
instruction  and  specialized  operational 
missions.  Of  particular  significance  in  such 
an  approach  to  training  is  a failure  to  con- 
ceptualize the  entire  system  in  perspective. 
This  results  in  a piecemeal,  poorly-coordi- 
nated buildup  of  instructional  units  with 
retrofitting  as  the  method  of  necessity  for 
curriculum  development.  Furthermore,  the 
Subject  Matter  Experts  (SMEs)  approaching  the 
task  usually  do  not  have  the  benefit  of  ex- 
pertise in  the  principles  and  application  of 
instructional  technology.  The  problem  is 
complicated  by  the  rapid  turnover  in  instruc- 
tor personnel.  Often  too,  an  attempt  is  made 
to  piece  together  the  in-house  training 
materials,  typically  instructor  guides  (out- 
lines), with  separately  developed  technical 
materials  provided  by  a contractor  to  comple- 
ment existing  hardware.  The  classroom  in- 
structor is  ultimately  left  with  the  task  of 
integrating  the  curriculum  while  performing 
within  the  confines  of  the  existing  policy 
and  procedure  of  the  training  command. 

With  the  limited  resources  and  increased 
demands  which  are  placed  upon  the  military 
training  community  today,  the  advent  of  a 
technology  of  instructional  systems  develop- 
ment has  demanded  more  attention  than  it 
might  have  in  more  affluent  times.  The 
challenge  for  greater  efficiencies  in  the 
learning  process  and  greater  cost-effective- 
ness in  the  training  program  has  been  recog- 
nized by  the  ROD  community  and  has  resulted 
in  the  derivation  of  numerous  instructional 
models.  Most  of  these  models,  which  are 
generally  oriented  toward  individualized 
instruction,  have  been  based  upon  a process 
leading  from  task  analysis  and  behavioral 
objectives  to  curriculum  development  and 
evaluation.  While  these  models  are  contin- 
ually undergoing  refinements,  the  need  is 
pressing  for  the  implementation  and  concurrent 
evaluation  of  the  basic  conceptual  framework 
of  the  Systems  Approach  to  Training  (SAT) 
within  the  operational  training  community. 


One  such  effort,  which  has  been  initiated 
within  the  Navy's  E-2C  aircrew  training  pro- 
gram, will  be  briefly  described  in  this 
paper. 


APPROACH 

The  Cal  span  Human  Factors  Section  has 
approached  the  problem  of  developing  an  E-2C 
SAT  program  (Sugarman,  et  a 1,1976)  by  coordi- 
nating the  skills  and  experiences  of  special- 
ists in  the  disciplines  of  psychology,  educa- 
tion, human  factors  engineering,  and  aviation 
Each  of  these  specialists  was  selected  for 
the  effort  on  the  basis  of  prior  operational 
and/or  training  experience  with  military  air- 
craft. In  addition,  one  Calspan  analyst  (a 
pilot/navigation-rated  engineer)  received 
additional  subject  matter  expertise  through 
participation  in  an  NFO  transition  course. 

The  general  Calspan  approach  to  SAT  programs 
is  to  put  together  such  a team  which  is  first 
and  foremost  a group  of  experts  in  SAT  tech- 
nology, but  whose  familiarity  with  the  sub- 
ject matter  is  significant  so  that: 

a.  high  rapport  can  be  established  with 
the  users; 

b.  subtleties  of  the  mission  require- 
ments are  not  overlooked; 

c.  operational  policy  and  constraints 
can  be  fully  appreciated; 

d.  total  reliance  is  not  placed  on  the 
availability  or  cooperation  of  the 
user's  subject  matter  experts;  and, 

e.  full  advantage  can  be  taken  of  the 
SAT  analysts' famil iarity  with  other 
relevant  missions  and  training  oper- 
ations. 

The  combination  of  this  type  of  SAT  analysts, 
who  know  the  questions,  with  an  ample  supply 
of  user  experts,  who  know  the  answers,  en- 
sures the  highest  quality  possible  for  the 
results  of  such  efforts. 

The  development  of  a comprehensive  data 
base  was  the  first  task  in  applying  the 
Calspan  SAT  methodology  to  the  E-2C  training 
program.  This  was  accomplished  through  a two- 
faceted  analysis.  One  facet  involved  an  ex- 
tensive analysis  of  the  existing  training 

"E-2C  Systems  Approach  to 
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program.  The  second  facet  consisted  of  a 
task  analysis  (Mitchell,  1975a;  Hinton, 

1975a).  The  background  information  for  the 
NFO  task  analysis  was  established  through 
firsthand  observation  and  recording  of  class- 
room presentations  conducted  by  squadron 
training  personnel.  While  this  appeared  to 
he  the  most  efficient  in  terms  of  obtaining 
information  about  the  NFO/FT  training, 
Calspan's  previous  experience  with  the  pilot/ 
co-pilot  training  task  and  the  availability 
of  documentation  (including  recently  developed 
audio-visual  presentations)  for  a significant 
portion  of  that  training,  alleviated  the 
necessity  for  a corresponding  approach.  In 
both  cases,  however,  all  available  training 
documentation  was  collected  for  analysis. 

These  informational  sources  were  then 
supplemented  by  informal  interviews  with 
squadron  training  personnel  and  direcc  obser- 
vations of  operational  equipment/ simulators 
and  crews.  In  this  procedure,  heavy  reliance 
was  made  on  two  sources,  subject  matter  ex- 
pertise and  operational  documentation.  These 
sources,  together  with  classroom  lecture 
notes  covering  operational  information  not 
otherwise  available,  provided  Calspan  with 
the  basic  information  for  preparing  the  task 
analysis  listing. 

The  format  which  was  employed  for  the 
task  analysis  was  one  adopted  from  experience 
with  a previous  training  program  design.  The 
resulting  data  base  consisted  of  a detailed 
hierarchical  breakdown  of  all  activities  re- 
quired to  perform  the  mission.  Of  special 
significance  were  the  level  of  detail  (task 
element  level),  the  mission-performance/ 
system-operation  orientation,  and  the  display- 
operator-control  relationships.  Furthermore, 
an  additional  effort  was  made  to  include  re- 
marks pertinent  to  task  performance  but  not 
directly  reflected  in  the  action  statement. 
Much  attention  was  given  to  this  effort,  as 
it  was  designed  to  provide  the  information 
necessary  to  formulate  behavioral  objectives 
and  to  provide  the  structure  of  the  lesson 
specifications  later  in  the  progTam. 

The  next  step  involved  the  identification 
of  training  objectives.  This  was  basically  a 
two-stage  process.  First,  the  task  analysis 
was  partitioned  into  a listing  of  discrete 
behavioral  components  that  formulated  the  be- 
havioral objectives  (Mitchell,  1975b;  Hinton, 
1975b).  Included  In  the  objectives  were  key 
features  such  as  ratings  of  performance  crit- 
icality and  difficulty,  and  the  specification 
of  performance  limits.  The  second  stage  in- 
volved the  "subtraction"  of  the  incoming 
skills  and  knowledges.  These  incoming  skills 
and  knowledges  for  various  trainees  were  es- 
tablished through  discussions  with  squadron 
personnel  as  well  as  an  analysis  of  documents 
related  to  their  previous  training.  Thus,  the 


behavioral  objectives  minus  the  incoming 
skills  and  knowledges  resulted  in  the  listing 
of  actual  training  objectives.  These,  too, 
were  later  used  in  the  preparation  of  lesson 
specifications  as  the  key  determinants  of 
content  areas. 

The  final  task  was  the  preparation  of 
lesson  specifications  (Fishburne,  et  al., 

1976;  Hinton  and  Blair,  1976).  As  alluded 
to  earlier,  each  of  the  previous  steps  con- 
tributed directly  to  the  product  of  this 
effort.  The  behavioral  objectives,  opera- 
tional policy,  availability  of  resources,  and 
incoming  skills  and  knowledges  played  a key 
role  in  the  instructional  design.  The  format 
of  the  lesson  specification  was  thus  divided 
into  three  continuous  domains--  cognitive, 
practice,  and  sortie/scenario.  In  the  cogni- 
tive domain,  the  sequence  of  instruction  was 
designed  to  present  in  a hierarchical  manner 
(building  a network  of  associations  upon  the 
developing  r pertoire)  that  information  which 
directly  contributed  to  mission  performance 
and/or  system  operation.  Accordingly,  the 
structure  of  the  training  objectives  was 
used  as  the  basis  for  the  teaching  points  in 
the  lesson  specifications.  The  ordering  of 
the  objectives  was  permuted,  however,  in  con- 
sonance with  learning  strategies. 

Integral  to  the  preparation  of  lesson 
specifications  were  the  media  analysis  and  the 
determination  of  instructional  strategies  and 
sequences.  The  specification  of  training  de- 
vices was  based  on  the  training  objectives 
identified  during  the  SAT  process  and  was 
oriented  toward  obtaining  the  best  possible 
trainee  performance  at  the  least  possible 
cost.  The  sequencing  of  instructional  blocks 
was  based  on  both  learning  principles  and 
effective  resource  (device)  utilization. 

These  two  efforts  are  detailed  in  the  para- 
graphs below. 


MEDIA  ANALYSIS 

Two  general  principles  were  followed  in 
determining  the  training  media  support  re- 
quirements for  the  pilot  and  NFO/FT  training 
courses.  The  first  principle  was  to  make 
the  maximum  possible  use  of  existing  media 
and  those  currently  under  procurement.  This 
involved  the  mapping  of  training  objective 
media  requirements  into  the  available  train- 
ing media  capabilities  to  evaluate  how  effec- 
tively the  training  objectives  identified 
during  the  SAT  process  can  be  taught  using 
those  media.  During  the  mapping  process, 
particular  emphasis  was  placed  upon  evaluating 
the  hands-on  capabilities  of  the  devices. 

The  second  principle  involved  a trade- 
off analysis  to  determine  the  need  for  the 
procurement  of  new  devices  or  modifications 
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to  existing  devices.  Factors  considered  in 
this  analysis  were  (1)  deficiencies  in  the 
current  devices  as  determined  by  the  mapping 
of  media  requirements  into  media  capabilities, 
(21  the  cost  of  procurement  of  new  devices, 

(3)  the  demand  on  existing  and  new  devices  as 
a function  of  trainee  flow  rate,  and  (4)  the 
degree  of  improvement  in  training  effective- 
ness to  ’-e  expected  by  procuring  new  devices 
or  modifying  existing  ones.  The  degree  of  im- 
provement expected  was  based  on  a considera- 
tion of  some  of  the  basic  principles  of  learn- 
ing, e.g.,  hands-on  learning  facilitates  the 
acquisition  and  retention  of  psychomotor 
skills;  individualized,  self-paced  instruction 
enhances  cognitive  learning;  and  testing  is 
most  effective  when  it  immediately  follows 
instruction  to  allow  timely  feedback  and  reme- 
diat ion. 

A comparison  of  the  stimulus  character- 
istics inherent  in  the  available  devices  with 
the  stimulus  characteristics  judged  by  SAT 
analysts  to  be  essential  for  the  training  of 
procedure-oriented  objectives,  indicated  that 
the  existing  training  devices  as  they 
are  currently  configured,  provide  ample 
capabilities  for  hands-on  learning.  The  main 
function  of  the  SAT  process  with  respect  to 
these  devices  was  to  ensure  their  effective 
utilization  as  a function  of  instructional 
strategies  and  sequencing. 

From  the  vast  plethora  of  educational 
research  that  has  been  conducted  in  recent 
years,  three  instructional  principles,  which 
increase  learning  speed  and  enhance  retention 
of  academic  material,  have  emerged.  These 
principles  involve:  (1)  student-paced  in- 

struction; (2)  individualized  presentation; 
and  (3)  immediate  knowledge  of  results.  The 
use  of  an  individualized,  self-paced  instruc- 
tional medium  accounts  for  individual  differ- 
ences in  learning  time  and  learning  style, 
and  when  supplemented  with  a testing  session 
at  the  end  of  each  lesson,  provides  the 
student  and  instructor  with  immediate  feedback 
of  student  performance.  This  immediate  know- 
ledge of  results  allows  the  student  to  review 
the  content  of  the  lesson  for  remediation 
while  lesson  content  and  problem  areas  are 
still  fresh  in  his  mind  and  also  assists  the 
instructor  in  offering  prompt  remediation 
guidance. 

The  results  of  this  SAT  analysis  are 
particularly  appropriate  for  implementation 
in  an  individualized  training  medium.  Through 
the  structured  steps  of  a SAT  analysis  only 
those  teaching  points  which  will  impinge  upon 
subsequent  trainee  performance  are  selected 
for  instruction.  Those  selected  teaching 
points  are  further  ordered  into  a lc;,.cal 
sequence  of  discrete  steps  which  can  be 
readily  converted  to  an  individualized  in- 
structional format.  Thus,  through  this 


selection  and  ordering  process  the  need  for 
student-instructor  interaction  is  minimized 
in  duration,  but  optimized  for  utility. 

Within  an  instructional  sequence  in 
which  the  teaching  of  psychomotor  and  cogni- 
tive skills  is  integrated  (see  next  section) 
the  implementation  of  an  individualized  sys- 
tem for  cognitive  instruction  allows  for  more 
efficient  use  of  student  time.  In  such  a 
system  the  training  technique  for  cognitive 
skills  is  consistent  with  that  for  psychomotor 
skills,  i.e.,  both  cognitive  and  psychomotor 
skills  are  taught  individually.  By  using  an 
individualized  approach  to  cognitive  training, 
as  well  as  to  psychomotor  training,  there  is 
much  more  flexibility  available  for  the 
scheduling  of  psychomotor  lessons,  because 
device  utilization  times  are  not  determined 
as  a function  of  times  when  classroom  lectures 
are  scheduled.  Rather,  the  scheduling  of 
training  devices  can  be  arranged  so  as  to 
make  the  most  efficient  use  of  all  available 
resources.  For  example,  with  an  integrated, 
totally  individualized  system  one  student  may 
be  engaged  in  cognitive  training  at  the  same 
time  another  student  in  the  same  convening 
class  is  engaged  in  psychomotor  training  in  a 
simulator. 

The  effectiveness  of  classroom  versus 
individualized  instruction  was  recently  com- 
pared for  two  Navy  training  courses,  the 
Aviation  Familiarization  Course  (AFAM)  and 
the  Aviation  Fundamentals  Course  (AMFU). 

The  results  indicated  that  learning  time  was 
30%  to  60%  less  for  the  individualized  in- 
struction, while  immediate  and  delayed  test 
performance  (retention)  was  essentially  the 
same  for  the  two  instructional  techniques. 

Individualized,  self-paced  instruction 
should  not  be  implemented  independent  of  a 
consideration  of  the  media  requirements  of 
each  training  objective.  For  example,  some 
objectives  are  amenable  to  self-paced  individ- 
ualized instruction,  while  other  objectives 
can  more  effectively  be  taught  in  a group 
setting. 

On  the  basis  of  an  evaluation  of  the 
subject  content  of  the  pilot  and  NFO/FT  cog- 
nitive training  objectives,  SAT  analysts  de- 
termined that  an  individualized,  self-paced 
instructional  medium  should  replace  the 
traditional  classroom  lecture  as  the  pre- 
ferred medium  for  cognitive  instruction.  To 
supplement  this  medium,  classroom  lecture/ 
discussion  was  retained  for  those  objectives 
which  require  intcraction/discussion  betwe  n 
instructors  and  students,  and  among  students. 


INSTRUCTIONAL  STRATEGY 

Strategies  within  instructional  blocks 
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in  which  cognitive  learning  takes  place  were 
presented  in  the  previous  section  (i.e.,  stu- 
dent-paced instruction,  individualized  pre- 
sentation, and  testing  with  immediate  know- 
ledge of  results).  Within  practice  and 
sortie/scenario  blocks  the  primary  strategy 
was  hands-on  learning  with  immediate  knowledge 
of  results  available  either  through  instruc- 
tor feedback  or  the  automated  performance 
measuring  capability  of  the  OFT.  Pacing  in 
practice  and  sortie/scenario  lessons  was  based 
on  instructor  evaluation  of  trainee  perfor- 
mance. 

Guidelines  used  in  the  sequencing  of  in- 
structional blocks  were  derived  from  the 
principles  of  learning.  These  guidelines  per- 
tain across  the  spectrum  of  training  objec- 
tives, and  when  implemented,  provide  for  the 
logical  ordering  of  instructional  blocks  to 
produce  an  efficient  and  cost-effective 
learning  sequence.  One  of  these  guidelines, 
which  was  discussed  above,  is  early  hands-on 
learning.  This  philosophy  of  active  partici- 
pation results  in  superior  training  effective- 
ness through  increased  attention  and  motiva- 
tion and  by  giving  the  trainee  the  opportunity 
to  integrate  cognitive  training  with  hardware 
experience. 

The  principle  of  presenting  new  informa- 
tion that  is  associated  with  previously 
learned  concepts  is  well-documented  in  the 
literature.  The  general  result  of  adhering 
to  this  principle  is  that  a context  is  pro- 
vided within  which  the  new  information  be- 
comes more  meaningful.  A term  used  within 
the  SAT  methodology  is  the  "enabling  objectives 
hierarchy."  Prerequisite  (enabling)  skills 
and  knowledges  are  usually  acquired  in  simple 
devices  (e.g. , a carrel)  and  are  later  inte- 
grated into  a more  complex  skill  in  a more 
complex  device  (e.g.,  the  Tactics  Trainer). 

By  interspersing  hands-on  lessons  among 
cognitive  lessons,  retention  of  both  the 
knowledge  and  the  skills  exercised  in  imple- 
menting the  knowledge  is  enhanced  (i.e.,  one 
learns  by  doing).  Practice  lessons  are 
spaced  within  the  course  sequence  so  that, 
whenever  possible,  the  knowledge  learned  in 
the  immediately  preceding  cognitive  lesson(s) 
is  implemented.  Sortie/scenario  lessons 
generally  build  on  the  preceding  practice 
lessons  and  provide  the  trainee  with  the 
opportunity  to  integrate  the  knowledge  and 
skills  acquired  in  the  cognitive  and  practice 
lessons. 

There  is  an  apparent  conflict  between  the 
sequencing  philosophy  in  which  device  utiliza- 
tion is  integrated  throughout  the  course  se- 
quence and  the  sequencing  philosophy  in  which 
the  complexity  of  devices  used  increases  as 
an  ordered  training  sequence  proceeds  from 
cognitive,  to  practice,  to  sortie/scenario. 


This  is  particularly  true  with  respect  to  air- 
craft utilization  during  pilot  training.  If 
the  building  philosophy  were  strictly  fol- 
lowed, aircraft  sorties  could  not  begin  until 
all  cognitive  training  and  early  hands-on 
training  in  ground-based  devices  are  com- 
pleted. Such  a sequencing  strategy  would  be 
based  on  the  argument  that  efficient  aircraft 
utilization  is  a function  of  the  completeness 
of  prior  cognitive  learning  and  practice. 

The  benefits  to  be  derived  from  integrating 
aircraft  sorties  into  the  total  training  se- 
quence is  superior  to  the  sequential  ground- 
then-flight  phase  approach  which  is  currently 
being  used  in  the  E-2C  training  program.  The 
rationale  for  integration  of  aircraft  sorties 
into  the  total  instructional  sequence  is 
based  upon  three  learning  guidelines.  One 
such  guideline  is  that  training  requiring  the 
most  practice  should  start  early  in  the  train- 
ing sequence.  Certainly  learning  to  control 
the  aircraft  in  a wide  variety  of  situations 
is  the  most  difficult  aspect  of  any  pilot 
training  program;  and,  therefore,  experience 
"at  the  wheel"  should  begin  early  in  the 
training  sequence. 

A second  guideline  relates  to  the  fact 
that  learning  and  retention  generally  in- 
crease as  the  fidelity  of  the  framework  in 
which  the  learning  is  used  increases.  That 
is,  as  knowledge  and  practice  in  the  ground- 
based  portions  of  the  training  sequence  are 
accumulated,  they  can  be  utilized  in  the 
"real-world"  environment,  thus  facilitating 
the  acquisition  of  new  skills  and  knowledge 
and  enhancing  the  retention  of  previously- 
learned  skills  and  knowledges. 

A third  guideline  involves  the  effect 
of  motivation  on  trainee  performance.  For  a 
pilot  trainee  there  is  no  motivational  sub- 
stitute for  allowing  him  to  do  what  he  came 
into  the  program  to  do,  fly  the  airplane. 

By  scheduling  sorties  throughout  the  training 
sequence,  trainee  motivation  will  be  enhanced 
during  all  portions  of  the  training  program. 
Motivational  considerations  also  apply  to  the 
integration  of  "real-world"  experience  into 
the  NFO  training  sequence;  however,  the  de- 
sirability of  a totally  integrated  sequence, 
in  which  the  E-2C  is  extensively  used,  is  re- 
duced due  to  the  high  degree  of  fidelity 
attainable  in  the  Tactics  Trainer. 

Fidelity  refers  to  the  "realism"  of  the 
simulation.  Even  though  the  specifications 
for  the  OFT  call  for  a high  degree  of  realism 
in  all  aspects  of  the  simulation,  integration 
of  sorties  into  the  training  sequence  is 
necessary  to  ensure  transfer  of  training  to 
the  aircraft,  and  for  instruction  in  phases 
of  flight  that  cannot  be  adequately  handled  in 
the  OFT.  The  high  fidelity  of  simulation  in 
the  Tactics  Trainer  results  in  the  capabili- 
ties for  the  simulation  of  tactical  situations 
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superior  to  that  in  the  aircraft.  For  this 
reason,  Tactics  Trainer  practices  and  scenar- 
ios are  integrated  into  the  instructional 
sequence  with  aircraft  missions  being  incor- 
porated toward  the  end  of  the  NFO/FT  training 
course.  These  aix-craft  missions  will  ac- 
quaint the  trainees  with  the  intricacies  of 
airborne  tactical  data  (ATDS)  operation  in 
the  "real -world"  environment  (e.g.,  radar 
presentations) . 

Another  guideline  that  results  in  in- 
creases in  trainee  motivation  and  facilitates 
learning  is  spaced  practice  (as  opposed  to 
massed  practice).  This  guideline  pertains  to 
both  material  and  devices.  That  is,  trainee 
boredom,  accompanied  by  a resultant  decrease 
in  performance,  can  result  from  a long  expo- 
sure to  the  same  material  or  a long  period  of 
time  in  the  same  device.  Hence,  every  oppor- 
tunity was  taken  to  ensure  that  the  student 
is  exposed  to  a varied  media  mix. 

The  guidelines  discussed  above  apply  as 
general  sequencing  rules  across  the  entire 
instructional  program  and,  on  a smaller 
scale,  to  sequencing  within  modules.  (A 
module  is  a series  of  lessons  grouped  by 
subject  content  area.)  The  sequencing  among 
modules  is  based  upon  the  factors  of  diffi- 
culty, criticality,  and  frequency  of  occur- 
rence. Fssentially,  difficulty  of  the  learn- 
ing involved,  criticality  of  the  learning  to 
mission  success,  and  the  probability  of 
occurrence  of  an  event  impact  upon  the  place- 
ment of  different  modules  within  the  overall 
course  sequence.  For  example,  instruction  on 
engine  operations  should  occur  early  in  the 
training  sequence  because  (1)  understanding 
engine  operation  and  learning  the  normal  and 
emergency  engine  procedures  is  a difficult 
topic  area,  (2)  proper  engine  operation  and 
responses  to  abnormalities  are  critical  to 
the  mission,  and  (3)  the  knowledge  of  engine 
operation  is  frequently  used  in  both  normal 
and  emergency  situations.  On  the  other  hand, 
instruction  on  the  automatic  flight  control 
system  (AFCS)  may  come  later  in  the  sequence 
because  (1)  operation  of  the  AFCS  is  rela- 
tively simple  (2)  the  AFCS  is  not  critical 
to  mission  performance,  and  (3)  the  training 
orientation  is  not  to  use  the  AFCS  during 
most  of  the  sorties. 

The  resultant  basic  structure  of  the 
pilot  and  NFO  training  sequences  is  one  of 
"systems  within  mission  context."  That  is, 
systems  knowledge  is  integrated  into  the 
sequence  as  the  trainee  is  learning  and  per- 
forming the  E-2C  mission.  As  discussed 
earlier,  this  approach  involves  early  hands- 
on  learning  in  which  the  trainee  first  learns 
operating  procedures,  then  practices  the  pro- 
cedures, and  finally  performs  the  procedures 
in  more  complex  situations  in  which  decisions 
and  integration  with  other  systems  procedures 


are  required. 


DISCUSSION 

The  individualized  instructional  system 
which  Calspan  has  specified  for  the  F.-2C  air- 
crew was  designed  to:  (1)  promote  learning 

efficiencies  through  self-pacing,  immediate 
feedback,  early  hands-on  training,  and  cri- 
terion referencing;  (2)  increase  instruc- 
tor effectiveness  through  direct  student  con- 
tact (one-to-one)  as  needed;  and  (3)  en- 
hance training  system  operations  through 
flexible  scheduling.  To  accomplish  these 
goals,  tape/slide  presentations  have  been 
selected  for  the  primary  instructional  media, 
with  self-scoring  answer  sheets  serving  the 
testing  function.  The  tape/slide  presenta- 
tions allow  for  standardization  of  the  curri- 
culum, multiple  sensory  stimulation,  and  re- 
lative ease  of  incorporating  changes.  The 
self-scoring  answer  sheets  provide  immediate 
feedback,  while  maintaining  a record  of  stu- 
dent performance  for  instructor  evaluation. 

This  approach  was  followed  in  the  development 
of  a computer  managed  instructional  system 
for  Navy  technical  training.  While  the  small 
student  flow  rate  of  the  present  and  pro- 
jected E-2C  program  does  not  warrant  computer 
support  at  this  time,  the  system  is  amenable 
to  that  additional  support,  should  it  become 
appropriate  in  the  future.  A low  cost  comput- 
er managed  instruction  (CMI)  terminal  for 
support  of  a small  learning  center  is  already 
on  the  market,  and  improvements  are  develop- 
ing rapidly.  While  impressive  advances  have 
also  been  made  in  other  technologies,  such  as 
computer  aided  instruction  (CAI),  the  clear 
advantage  of  CMI  is  its  cost-effectiveness 
(Hansen  and  Fishburne,  1975).  Moreover,  the 
lack  of  definite  empirical  data  demonstrating  en- 
hanced learning  through  direct  interaction 
with  the  computer  (CAI),  point  toward  a 
greater  justification  for  the  recommended 
approach  for  handling  an  increase  in  trainee 
flow  rate. 

In  developing  the  lesson  specifications 
to  support  an  individualized  instructional 
system  for  the  E-2C,  the  traditional  steps  of 
the  ISD  process  were  followed,  but  with  a 
greater  emphasis  on  a "system"  orientation 
and  an  appreciation  for  the  interactive  na- 
ture of  the  developmental  steps.  Operational 
policy  and  economic  data  have  contributed  to 
the  selection  of  training  devices  and  their 
scheduling,  as  well  as  to  training  priorities. 
These  factors  have  been  carefully  weighed 
against  the  data  from  the  training  analysis, 
especially  the  task  analysis. 

The  task  analysis  employed  in  the  E-2C 
SAT  was  heavily  oriented  toward  operational 
documentation  and  was  followed  to  the  level 
of  discrete  perceptual -motor  actions.  This 
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approach  was  adopted  for  two  hasic  reasons. 
First,  it  resulted  in  a mission-related  basis 
for  the  analysis,  while  ensuring  that  the 
necessary  "conceptual"  systems  information 
was  documented  in  the  data  base.  Conse- 
quently, it  was  able  to  encompass  many  im- 
portant operations  which  would  otherwise  have 
been  overlooked.  Secondly,  it  served  to  educate 
the  instructional  psychologists  in  the  F.-2C 
subject  matter  domain.  The  Calspan  team  was 
then  able  to  expand  and  verify  this  initial 
data  base  through  interactions  with  E-2C 
training  personnel. 

The  behavioral  objectives  were  developed 
from  the  task  analysis  in  a format  that 
greatly  expanded  upon  the  traditional  state- 
ment of  behavior,  condition,  and  standard. 

The  additional  effort  required,  however, 
greatly  contributed  to  the  system  orientation 


and  was  directly  applicable  to  the  evaluation 
conditions  and  criteria.  The  mission/system 
orientation  of  the  task  analysis  was  main- 
tained in  the  behavioral  objectives  as  a pre- 
caution against  the  inclusion  of  unnecessary 
training  objectives. 

Finally,  the  lesson  specifications  were 
developed  in  a manner  directly  traceable  to 
the  task  analysis  and  hehavioral  objectives. 
Traditionally,  it  has  been  at  this  point 
that  other  ISO  efforts  depart  from  their 
origin.  In  the  present  procedure,  however, 
an  effort  was  made  to  ensure  continuity. 

Using  a continuum  from  cognitive,  to  practice 
and  sortie/scenario  lessons,  a performance 
orientation  of  training  objectives  and  subse- 
quently, lesson  segments  was  assured.  All  of 
this  has  been  presented  in  a format  consis- 
tent with  forthcoming  instructional  material. 


REFERENCES 

Fishburne,  Robert  P. , Jr.,  Mitchell,  John  F. , 
Johnson,  Steven  L. , and  Blair,  AnndreaJ. 

E-2C  CICO/ACO/FT  l.esson  Specifications. 

Calspan  Report  (Contract  No.  N61339-75-C-0101 ) , 
May  1976. 


Hansen,  Duncan  N.  and  Fishburne,  Robert  P. , 
J r . The  Operation  of  Computer  Managed  In- 
struction in  the  Navy:  Current  and  Future 
Perspectives.  Proceedings  of  the  8th  NTEC./ 
Industry  Conference,  November  1975. 


Hinton,  William  M. , Jr.  E-2C  Pilot  and  Co- 
pilot Task  Listing.  Calspan  Report  (Contract 
No.  N61 3 39-7 5 -C- 01 01 ) , September  1975a. 


Hinton,  William  M.  , Jr.  F.-2C  Pilot  and  Co- 
rd lot  Behavioral  Objectives.  Calspan  Report 
(Contract  No.  N61339-75-C-0101) , November 
1975b. 


Hinton,  William  M. , Jr.  and  Blair,  Anndrea  J. 
F.-2C  Pilot  and  Co-pilot  Lesson  Specifications. 
Calspan  Report  (Contract  No.  N61339-75-C-0101) , 
May  1976. 

Mitchell,  John  F.  E-2C  NFO/FT  Task  Listing. 
Calspan  Report  (Contract  No.  N61339-75-C- 
0101),  October  1975a. 

Mitchell,  John  F.  E-2C  NFO/FT  Behavioral 
Objectives.  Calspan  Report  (Contract  No. 
N61339-75-C-0101) , November  1975b. 

Sugarman,  Robert  C.,  Johnson,  Steven  L. , 

Hinton,  William  M.,  Jr.,  and  Buckenmaier, 
Chester  C.  SAT  Revisited--A  Critical  Post- 
examination of  the  Systems  Approach  to 
Training.  Paper  presented  at  the  19th 
Annual  Convention  of  the  Human  Factors 
Society,  Dallas,  October  1975. 

Sugarman,  Robert  C.,  Johnson,  Steven  L. , 
Mitchell,  John  F. , Hinton,  William  M. , Jr., 
and  Fishburne,  Robert  P. , Jr.  c-2C  Systems 
Approach  to  Training,  Phase  I Final  Report 
(Preliminary  Draft),  May  1976. 


ABOUT  THE  AUTHORS 

PR.  WILLIAM  M.  HINTON,  JR.  i*  a Research  Psychologist  in  the  Human  Factors  Section 
at  Calspan  Corporation  in  Buffalo , New  York.  He  is  involved  in  the  application  of 
Systems  Approach  to  training  technique*  for  the  B- ) and  E-2C  aircrew  training  pro- 
gram*. Prior  expedience  include*  military  and  commercial  helicopter  flying,  part- 
time  umvencty  teaching,  and  research  in  psychomotor  training  technique*.  He 
hold*  M.S.  and  Ph.V.  degree*  in  Industrial  Engineering  (Human  Factors)  i (tom  the 
State  University  of  New  Vork  at  Buffalo. 

PR.  ROBERT  P.  FISHBURNE,  JR.  i*  a Research  Psychologist,  with  Calspan  Corporation’ s 
Human  Factor*  Section  in  Buffalo,  dew  York.  He  has  more  than  five  year*  of  experience 
in  military  psychology,  three  0($  which  were  spent  a*  a Naval  aeorspace  experimental 
psychologist.  Since  joining  Calspan  in  1 975,  he  ha*  been  a principal  investigator 
m the  Systems  Approach  to  training  for  the  E-2C  aircrew.  He  hold*  an  M.A.  degree 
in  Experimental  Psychology  from  Georgia  Southern  College  and  an  Ed. V.  degree  in 
Curriculum  and  Instruction  (Research  Methodology  and  Statistics)  from  Memphis  State 
University. 


178 


ACTION  SPEED  TACTICAL  TRAINERS 

CAPTAIN  R.  H.  GRAHAM,  ROYAL  NAVY  (RETIRED) 
Ferranti  Limited 


INTRODUCTION 

This  paper  demonstrates  the  requirement 
for  a shore-based  Naval  tactical  trainer;  out- 
lines the  types  of  training  that  can  be  con- 
ducted and  the  disciplines  in  which  training 
can  be  given;  explains  how  the  trainer  can  be 
used  for  tactical  investigations;  lists  the 
principal  features  of  a low  cost  ASTT ; and 
concludes  with  a short  example  of  a tactical 
game . 

Requirements  for  Shore-based  Naval  Tactical 
Trainers 

Navies  need  a shore-based  Naval  tactical 
training  facility  or  an  Action  Speed  Tactical 
Trainer  (ASTT)  because  of: 

a.  Restrictions  imposed  on  fleet  move- 
ments with  the  high  cost  of  fuel  oil. 

b.  The  tremendous  flexibility  of  the 
shore-based  trainer  and  the  facility  with 
which  the  instructors  can  control  the  training 
environment . 

In  a single  forenoon  in  the  ASTT,  students 
can  be  given  experience  in  facing  a wide 
variety  of  differing  tactical  problems  such 
as  they  might  not  meet  in  a whole  week  of  sea 
exercises.  Furthermore,  this  experience  can 
be  extended  to  include  situations  which  can- 
not normally  be  reproduced  at  sea;  for 
instance,  a wave  attack  by  air  and  submarine- 
launched  missiles. 

Captain  Stephen  Roskill,  in  his  book  "The 
Strategy  of  Sea  Power,"  draws  frequent 
attention  to  the  failure  of  Navies  to  study 
and  practise  tactics  during  the  Intervals  of 
peace.  To  quote  just  one  example:  in  a 

reference  to  the  situation  in  the  Royal  Navy 
between  the  two  world  wars,  he  writes: 

"Trade  protection  was  still  regarded  as 
being  chiefly  a matter  of  locating 
enemy  surface  raiders  and  bring  them 
to  book.  Nor  were  any  exercises  carried 
out  to  investigate  the  problems  of  using 
aircraft  as  well  as  flotilla  vessels  to 
defend  mercantile  convoys  against  sub- 
marine attacks.  Co-operation  with  the 
shore-based  aircraft  of  the  Royal  Air 
Force  was  based  on  the  priorities 
established  for  ship-borne  aircraft,  and 
none  of  the  former  received  any  training 


in  antisubmarine  tactics,  in  the 
defence  of  convoys,  or  in  attacks  on 
enemy  merchant  shipping.” 

Of  course,  there  can  be  no  substitute  for 
sea  training;  but  I believe  there  is  a very 
real  need  for  it  to  be  complemented  by  shore- 
based  tactical  training.  And  this  shore-based 
training  needs  to  be  introduced  at  regular 
intervals  throughout  the  career  of  all  line 
officers . 

Let's  look  a little  more  closely  at  the 
type  of  training  I have  in  mind.  Broadly 
speaking,  operational  training  can  be  divided 
into  three  categories: 

First,  Operator  training,  when  we  train 
the  individual  operator  in  his  basic 
skills  as,  for  instance,  a radarman.  I 
am  not  talking  about  that. 

Second,  procedural  training,  when  we 
train,  for  instance,  a Cl C crew  in  their 
collective  responsibilities  for  running 
a complex  CIC.  I am  not  talking  about 
that . 

The  third  and  highest  category  is  the 
training  of  individuals  and  command 
teams  in  the  tactical  control  of  the 
maritime  forces  under  their  command: 
the  selection  of  Task  Group  formations 
and  screens;  redeployment  in  the  face  of 
differing  threats;  the  dispatch  of 
Surface  Attack  Units  - when,  where, 
whom?;  radio  and  radar  silence  policies; 
the  tactical  deployment  of  weapons  and 
the  use  of  sensors.  This  is  the  style 
of  training  for  which  an  ASTT  is 
designed. 

In  the  Royal  Navy,  the  ASTT  is  part  of 
the  Maritime  Tactical  School.  This  has  the 
great  advantage  that  lectures  on  tactical  sub- 
jects can  be  interspersed  with  practical  ex- 
perience on  the  trainer.  Furthermore, 
student  syndicates  can  be  invited  to  prepare 
their  own  operation  orders  to  meet  a partic- 
ular situation,  and  later  to  put  these  orders 
into  practice  on  the  trainer,  where  the  short- 
comings quickly  become  apparent.  Subsequent- 
ly, they  must  explain  their  decisions  during 
the  debriefing  session.  This  is  the  real  way 
in  which  lessons  are  learnt  and  ideas  are 
developed . 
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There  are  two  important  requirements 
in  an  ASTT.  (1)  It  should  be  inexpensive, 
and  (2)  it  should  be  versatile,  so  that  as 
new  ships,  aircraft,  weapons  or  sensors  are 
designed,  or  as  new  intelligence  on  potential 
enemy  equipment  becomes  available,  these  ships 
and  aircraft  may  be  correctly  simulated  in 
the  trainer  without  modification  of  the  ASTT 
hardware  or  software. 

In  a typical  ASTT  students  operate  in 
cubicles,  like  the  one  in  the  Royal  Navy's 
ASTT  at  the  Maritime  Tactical  School,  H.M.S. 
Dryad.  The  cubicle  is  fitted  with  general- 
purpose  displays  so  that  it  may  represent  any 
type  of  ship  or  aircraft.  No  attempt  is  made 
to  fit  the  cubicle  with  the  specific  equip- 
ment which  is  found  in  a ship's  Combat 
Information  Center  (CIC).  The  cost  of  doing 
this  plus  the  cost  of  stimulating  all  the 
different  displays  and  readouts  would  defeat 
our  fundamental  requirement  of  low  cost;  and, 
in  any  case,  we  are  training  at  a higher  level, 
a level  which  uses  as  a tool  in  decision- 
making the  picture  which  has  been  compiled 
and  presented  in  a ship's  CIC. 

We  use  these  displays  to  show  the 
student  the  tactical  situation  as  he  would 
see  it  in  real  life  at  sea,  taking  into 
account  the  manner  in  which  he  has  decided 
to  use  his  various  sensing  devices.  If  he 
has  chocen  to  keep  radar  silence,  then  he 
will  be  denied  target  information  which  is 
derived  solely  from  active  radar.  This  is 
done  automatically  by  the  computer.  And, 
of  course,  the  computer  does  a lot  more. 

When  assessing  which  radar  targets  would 
be  detectable  by  any  one  cubicle,  it  takes 
into  account:  the  radar  type;  the  vertical 
coverage  diagram,  including  horizon 
limitations;  any  sector  sweep  which 


Students  in  one  of  the  cubicles  of  the 
Action  Speed  Tactical  Trainer  at  HMS  DRYAD 


may  have  been  ordered;  and  the  effect  of 
enemy  jamming. 

Similarly,  for  sonar,  the  computer  takes 
into  account:  the  sonar  type;  the  sector 

being  scanned;  the  depth  of  the  thermal  layer 
and,  because  sonar  detections  are  never  as 
consistent  as  radar  detections,  a probability 
factor  which  is  related  to  a number  of  vari- 
ables, including  own  ship's  speed. 

To  enable  the  student  to  control  his 
ship  or  aircraft,  he  is  provided  with  a key- 
board with  a large  number  of  special  purpose 
keys  which  he  uses  to  alter  course,  switch  on 
or  off  sensors,  fire  weapons,  release  decoys, 
up  periscope,  etc.  The  capability  of  his 
craft  is  stored  in  the  computer  memory  thus 
ensuring  that  he  turns  with  the  the  correct 
tactical  diameter;  the  targets  he  detects  are 
those  which  are  detectable  by  his  own  sensors 
and  he  cannot  go  on  firing  after  he  has  ex- 
pended his  outfit  of  ammunition. 

Also  in  the  cubicle  is  a set  of  communi- 
cations terminals.  Radio  nets  are  simulated, 
with  computer  controlled  range  limiting  of 
UHF  channels.  Data  links  are  simulated  by 
allowing  all  ships  and  aircraft  on  the  net 
and  who  are  within  radio  range  to  receive  the 
transmitted  track  data.  Command  and  control 
orders  could  be  added  if  required.  Under- 
water telephone  is  also  simulated.  And 
finally,  there  are  internal  communications  in 
the  form  of  intercom  to  the  controllers  in 
the  Control  Room,  each  of  whom  represents  the 
different  out-stations  for  one  or  more 
cubicles. 

The  number  of  cubicles  fitted  in  an  ASTT 
depends  largely  on  the  number  of  students  you 
expect  to  train  at  any  one  time.  The  Royal 
Navy  has  twenty;  some  trainers  have  as  few  as 
six.  The  point  here  is  that  you  don't 
require  a cubicle  for  each  track  in  the  game 
- you  could  never  hope  to  man  them  all.  So, 
in  a typical  small  ASTT,  you  might  find  eight 
surface  ships  represented  by  the  eight 
cubicles.  Then  we  use  computer  generated 
tracks,  or  supplementary  tracks,  as  we  call 
them,  to  represent  all  the  enemy  ships  and 
aircraft,  and  all  the  missiles,  torpedoes 
and  other  detectable  weapons  which  may  be 
fired  during  the  game.  These  supplementary 
tracks  are  under  the  control  of  the  in- 
structional staff  and  can  be  made  to  man- 
oeuvre, operate  sensors  and  fire  weapons  as 
they  would  do  in  real  life.  They  are  a very 
inexpensive  way  of  providing  a full  and 
realistic  tactical  environment. 

Now  let's  look  at  the  facilities 
provided  for  the  instructional  staff,  or 
Controllers.  The  Royal  Navy's  ASTT  has  a 
control  room  and  a separate  debriefing  room. 


but  in  smaller  ASTTs  it  is  often  found  econ- 
omical to  combine  these  into  a single  room. 
This  makes  sense,  because  the  requirements  of 
the  two  rooms  are  in  many  ways  similar. 


Controlling  a simulated  exercise  in  the 
Action  Speed  Tactical  Trainer  at  KMS  DRYAD 

Now,  the  principal  tasks  of  the  Control- 
lers are  to  control  the  tempo  and  the  running 
of  the  game  and  to  supervise  the  students. 
Secondary  tasks  are  to  control  the  supple- 
mentary tracks,  represent  out-stations, such 
as  the  bridge,  etc.,  and  to  impose  casualties. 
So  it  is  important  that  controllers  are 
relieved  of  all  procedural  chores,  such  as 
assessing  who  can  detect  which  targets,  or 
guiding  a missile  along  a predetermined 
flight  path.  This  of  course  is  where  the 
computer  comes  in.  As  in  any  well  designed 
system,  it  is  made  to  perform  all  the 
functions  which  do  not  require  human  judge- 
ment, such  as  the  movement  of  all  tracks  in 
accordance  with  their  ordered  courses  and 
speeds,  the  detectability  calculations  to 
which  I have  already  referred  and  the  pro- 
cessing of  information  for  showing  on  the 
displays  in  accordance  with  the  operator's 
requi rement s . 

Controllers  must  of  course  be  provided 
with  display  consoles  with  keyboard  and 
tracker  ball.  This  was  the  console  arrange- 
ment for  the  DRYAD  ASTT,  hut  subsequent 
developments  in  display  technology  have 
allowed  this  26-inch  tube  to  be  used  as  a 
combined  labelled  plan  display  and  tote. 

This  is  very  much  liked  by  controllers  since 
it  provides  all  the  information  they  require  on 
on  a single  display. 


As  I have  said,  it  is  important  that 
controllers  are  able  to  supervise  the  actions 
of  the  students.  To  enable  them  to  form 
sound  judgements,  it  is  important  that  they 
are  able  to  see  the  tactical  situation  as 
this  is  known  to  the  students  in  the  cubicle. 
This  they  can  do  by  switching  their  control 
display  to  .show  the  picture  as  seen  in  any 
selected  cubicle. 

Whilst  their  display  is  so  switched;  or, 
indeed,  if  they  choose  to  suite h their  dis- 
play to  a short  range  scale  to  examine  some 
situation  in  greater  detail,  they  will  be 
blind  to  the  overall  game  situation.  So 
there  is  provided  at  one  end  of  the  control 
room  a large  screen  display,  visible  to  all 
controllers,  on  which  is  shown  the  complete 
game  situation,  with  all  tracks  moving  in 
real-time.  There  are  many  types  of  large 
screen  displays  on  the  market  -one  popular  in 
U.K.  at  the  moment  is  this  one  which  uses 
Schmidt  optics  principles  and  which  can  be 
used  for  monochrome  or  multi-colour  pro- 
jection. With  multi-colour,  there  may  be  a 
small  problem  of  registration  at  the  extrem- 
ities of  the  display.  Otherwise,  it  has 
tremendous  attractions. 

Another  advantage  of  the  large  screen 
display  is  that  spectators,  and  indeed 
students  from  cubicles  which  have  been  sunk, 
can  watch  the  progress  of  the  game  without 
inconveniencing  the  controllers. 

And  of  course  the  large  screen  display 
is  essential  for  debriefing,  which  explains 
why  I mentioned  earlier  that  it  is  often 
found  convenient  to  combine  the  control  and 
debriefing  rooms  into  a single  room.  In  fact, 
the  only  reason  for  requiring  two  rooms  is  to 
allow  debriefing  of  one  game  to  run  con- 
currently with  the  playing  of  another. 

Debriefing  is  a very  important  aspect  of 
tactical  training.  This  is  the  occasion  when 
students  must  stand  up  and  account  for  their 
decisions,  when  discussion  stimulates  thought, 
and  when  lessons  are  learnt  rather  than  just 
heard.  To  provide  a background  against  which 
these  dicussions  can  develop,  and  to  provide 
a pictorial  record  of  exactly  what  happened 
so  that  students  can  see  where  they  went 
wrong,  the  large  screen  display  is  needed. 

During  the  playing  of  the  game,  the  move- 
ment of  all  tracks  is  recorded  so  that  the 
game  may  be  replayed  on  the  large  screen 
display  for  debriefing.  It  is  normal  to  be 
able  to  rerun  the  game  in  real-time  or  at 
speeds  up  to  sixteen  times  faster  so  as  to 
pass  quickly  over  the  dull  periods;  and  also 
to  be  able  to  stop  at  any  selected  time  in 
order  to  discuss  a situation  of  particular 
tactical  interest. 


The  trainer  can  also  be  used  for  tactical 
evaluations.  I.et  uS  suppose  one  wants  to  test 
the  operational  concept  for,  and  indeed  ti.e 
value  of,  a small  aircraft  carrier  carrying 
a mixed  group  of  VTOL  aircraft  and  helicopters 
in  differing  operational  scenarios.  The 
staff  of  the  tactical  school  prepares  the 
scenarios  and  experienced  officers  from 
Fleet  or  Squadron  staff,  or  from  the  school, 
prepare  their  operational  plans  and  then  man 
the  cubicles  and  fight  it  out.  The  same 
scenario  can  be  played  a number  of  times 
with  different  officers  taking  different 
responsibilities.  From  this,  a clear  lesson 
almost  always  emerges.  Similarly,  tactical 
procedures  can  be  developed.  For  instance,  a 
small  Navy  not  fortunate  enough  to  own  a 
nuclear  submarine  can  develop  tactics  for 
containing  and  attacking  these  difficult 
targets.  One  is  not  restricted  to  ships  which 
already  exist.  The  computer  memory  can  be 
fed  with  the  characteristics  of  a vessel 


not  yet  on  the  drawing  board,  or  one  from  a 
bygone  age:  the  allocated  cubicle  will  have 

the  capability  exactly  as  defined,  and  the 
information  is  entered  as  data,  not  as  a 
program,  so  there  is  no  risk  of  consequential 
programming  problems. 

Of  course,  the  ASTT  can  be  used  for  any 
type  of  tactical  training,  be  it  Carrier  Task 
Force,  ASW,  convoy  work,  amphibious,  etc. 

CONCLUSION 

So  often  we  have  entered  a war  ignorant 
of  or  ignoring  the  tactical  lessons  learnt, 
often  at  great  cost,  in  previous  conflicts. 

I believe  very  strongly  that  an  ASTT  along 
the  lines  which  I have  described  provides  an 
inexpensive,  versatile  and  very  effective  tool 
to  ensure  that  we  never  again  fail  in  this 
aspect  of  our  Naval  training. 
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SIMULATION  PROCUREMENT  MANAGEMENT  PROBLEMS  AND  PERSPECTIVES 
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Office  of  the  Chief  of  Naval  Operations 


"Our  job  is  to  fool  you."  This  state- 
ment by  a senior  executive  of  a simulator 
manufacturing  firm  neatly  summarizes  the  ac- 
cepted essence  of  training  through  simula- 
tion. Such  a succinct  statement,  however, 
tends  to  mask  the  enormity  of  the  task  of 
synthesizing  the  training  environment  real- 
istically enough  to  ensure  a positive  trans- 
fer of  training,  while  keeping  costs  within 
reason.  Industry  has  risen  to  the  techno- 
logical challenge  by  developing  training 
media  which,  for  the  most  part,  faithfully 
perform  according  to  our  specifications. 
Although  a portion  of  the  quantum  jump  in 
training  technology  resulted  frrm  a wartime 
economy  and  the  national  space  effort,  most 
military  applications  were  fostered  by  the 
energy  crisis.  In  1974  and  1975,  there  was 
an  unprecedented  infusion  of  funds  into  our 
simulator  acquisition  program.  During  this 
period,  the  Congress  came  to  the  realization 
that  simulators  could  be  used  for  substitu- 
tion of  flying  hours,  rather  than  simply 
supplementing  them  as  in  the  past.  The  air- 
lines had  been  doing  it  for  years!  Why  was 
the  military  so  far  behind?  All  required 
forces  were  not  acting  simultaneously  to 
create  the  utilization  paradox  that  presently 
confronts  simulator  procurement  managers. 

FLIGHT  HOUR  SUBSTITUTION 

The  present  generation  of  high-perform- 
ance, super-complex,  multimission  aircraft 
demands  more  hands-on  training  than  ever 
before,  while  at  the  same  time  flying  hours 
are  being  reduced  to  conserve  resources.  The 
solution  is  obvious:  use  simulators  as  a re- 

placement for  flying  hours.  From  the  know- 
ledge that  simulators  could  be  used  for  flight 
hour  substitution  grew  the  dictum  that  simu- 
lators would  be  used  for  flight  hour  substi- 
tution. 

The  requirement  for  flying  hour  substi- 
tution embodied  the  need  for  faithful  repli- 
cation of  the  training  environment  and  this 
translated  itself  into  higher  procurement 
and  support  costs.  The  aircraft  interior  and 
its  interactive  forces  had  to  Be  fully  and 
faithfully  replicated  to  justify  substitution. 

The  budget  analysts  at  the  Defense  and 
Congressional  levels  soon  realized  that  the 
cost  of  flying  hours  saved  as  a result  of  sim- 
ulator training  could  be  used  to  amortize  the 


cost  of  the  simulators.  The  next  step  was 
unfortunate  but  inevitable:  budgetary  con- 

siderations began  to  take  precedence  over 
t-aining  considerations  until  now  a procure- 
ment is  approved  or  disapproved  solely  on 
the  basis  of  its  amortization  period. 

TRAINING  EFFECTIVENESS 

Training  program  sponsors  are  asked  by 
the  user,  and  directed  by  the  Armed  Services 
Committee,  to  acquire  on  the  basis  of  trai  n- 
inq  effectiveness.  Their  primary  concern  is 
the  maintenance  of  fleet  readiness.  Budget 
analysts  and  the  Appropriations  Committees, 
on  the  other  hand,  approve  programs  pri- 
marily on  the  basis  of  cost  effectiveness. 
Figure  1 depicts  the  problem: 


The  greater  the  number  of  flying  hours  sub- 
stituted by  a trainer,  the  greater  its  like- 
lihood of  survival  at  the  budget  table.  There 
are  cost-effective  devices  and  there  are 
training-effective  devices,  and  very  often 
they  have  little  in  common.  Fortunately, 
there  are  a sizable  number  of  training  devices 
in  the  field  and  yet  to  come  that  *.re  solidly 
located  in  the  effectiveness  intersection. 

These  are  the  devices  that  are  presently  cap- 
able of  surviving  the  budget  process.  Further, 
unless  opinions  change  dramatically,  only  those 
devices  which  are  in  the  intersection  will  be 
worth  the  effort  required  to  sponsor  them  in 
the  budget  arena.  Clearly,  the  objective  of 
the  training  sponsor  must  be  to  increase  the 
size  of  the  overlap;  increase  training  ef- 
fectiveness and  reduce  costs  to  the  point 
where  a device  will  be  in  the  intersection 
and,  thus,  fully  effective. 

The  odd  relationship  of  simulator  train- 
ing to  the  rest  of  the  defense  program  is 
shown  in  figure  2. 
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Strategic  capability  rests  on  a foun- 
dation of  tactical  capability,  and  both  grow 
from  training.  Aside  from  confrontation, 
the  military  is  constantly  training  to  be 
fally  prepared  to  win  those  confrontations. 
Although  training  is  only  a rather  small  ele- 
ment of  defense  budget,  it  is  the  only  ele- 
ment expected  to  pay  for  i tsel f ! Trainers 
which  were  previously  considered  essential 
to  fleet  training  in  the  safe  and  effective 
operation  of  combat  aircraft  have  been  dis- 
approved by  reviewing  authorities  because 
they  do  not  amortize  well!  B-52,  C-5A, 

F-14,  and  S-3  aircraft  are  not  expected  to 
amortize,  but  the  trainers  for  these  weapons 
systems  must  do  so,  or  risk  being  cut  at  the 
budget  table. 

These  are  the  facts  of  life,  and  we  must 
learn  to  live  with  them,  at  least  for  the 
time  being.  It  is  not  our  purpose  to  bemoan 
the  passing  of  the  loose  money  era,  but  to 
propose  methods  by  which  cost  effectiveness 
may  be  increased  while  simultaneously  im- 
proving or,  at  the  very  least,  not  decre- 
menting training  effectiveness. 

INSTRUCTIONAL  SYSTEMS  DEVELOPMENT  (ISO) 

A step  in  the  right  direction  is  the  con- 
cept of  Instructional  Systems  Development 
USD).  Of  all  the  jargon  and  acronyms  in  our 
inventory,  certainly  ISD  is  the  current  lead- 
er in  misuse.  ISD  is  not  a product;  it  is  a 
process.  One  does  not  deliver  an  ISD;  one 
delivers  an  instructional  system  as  a product 
of  the  ISO  process.  Above  all,  one  does  not 
develop  an  ISD.  Theoretical ly , the  ISD  pro- 
cess is  one  by  which  we  may  identify  training 
objectives  and  deliver,  or  at  least  recommend, 
a homogenous  training  package  that  will  en- 
able us  to  meet  those  objectives. 

In  practice,  however,  we  have  paid  our 
money  and  taken  our  chances.  To  date,  the 
ISD  process  has  produced  everything  from  "lit- 
erature searches,"  which  have  been  little 
more  than  cut-and-paste  exercises  using  the 
GSA  catalogs,  to  scholarly,  innovative  stu- 
dies, proposing  a skein  of  media  that  would 


cost  several  fortunes  to  acquire.  We  are 
learning  though,  and  we  are  narrowing  the 
gap  between  what  is  desired  and  what  is  de- 
livered. 

Among  the  things  that  ISD  is  trying  to 
tell  us  are  some  obvious  ones,  such  as  it  is 
more  cost  effective  to  train  several  students 
or  crews  in  part-task  trainers  than  to  train 
a single  student  or  crew  in  a full-blown 
Weapons  Systems  Trainer  (WST)  on  a task  that 
only  uses  a small  percentage  of  its  capabil- 
ities. ISD  may  reinforce  the  need  to  have  a 
WST  for  total  mission  training,  but  almost 
certainly  it  will  also  tell  us  that  it  is 
neither  cost  nor  training  effective  to  meet 
only  limited  training  objectives  using  a ful- 
ly capable  WST.  In  reflection,  ISD  may  act- 
ually be  nothing  more  than  the  application  of 
training  common  sense.  ISD,  in  any  event,  is 
the  process  by  which  we  may  well  demonstrate 
training  effectiveness  as  well  as  cost  effec- 
tiveness. 

COST  EFFECTIVENESS 

There  is  no  reason  to  assume  that  fuel 
costs,  or  any  other  expenses  for  that  matter, 
will  decrease,  so  the  cost  per  hour  of  air- 
craft operation  will  undoubtedly  continue  to 
increase.  While  this  increase  makes  more  at- 
tractive the  lower  operating  costs  of  simula- 
tors, it  may  not  be  enough  to  support  accep- 
table amortization.  Even  with  increased  air- 
craft operating  costs,  the  only  thing  that 
will  guarantee  a trainer's  cost  effectiveness 
is  a dramatic  reduction  in  present  day  pro- 
curement and  support  bills.  We  can  hear  you 
now,  "You  can't  get  there  from  here!"  True 
enough!  True,  at  least  without  a radical  de- 
parture from  "business  as  usual."  Thatxdepar- 
ture  is  precisely  what  this  paper  proposes. 

It  will  be  no  revelation  to  most  to  state 
that  much  of  the  high  cost  of  acquisition  and 
support  can  be  laid  at  the  doorstep  of  the 
Government.  Over  the  years  we  have  developed 
a network  of  Military  Standards  that  have,  in 
many  cases,  little  or  no  effect  on  hardware 
capability.  But  they  have  a marked  effect  on 
soaring  acquisition  and  support  costs.  As 
Senator  Lawton  Chiles  has  pointed  out,  you  can 
build  a better  mousetrap,  but  you  won't  be  able 
to  sell  it  to  the  Government  for  less  than 
twenty  dollars!  And  in  the  halcyon  days  of 
unlimited  spending,  we  probably  would  have 
bought  it  for  twenty  dollars! 

MILSPECS 

Let  there  be  no  mistake  about  what  we  are 
saying;  Military  Standards  have  been  developed 
in  a systematic  way  to  help  guarantee  the  best, 
safest,  and  most  reliable  piece  of  equipment 
possible.  The  successful  prosecution  of  modern 
warfare  demands  reliability,  and  Military  Stan- 


184 


dards  provide  the  means  to  ensure  that  end. 

Our  point  is  this:  except  where  safety  is  a 

factor.  Military  Standards  have  only  limited 
application  to  trainers  that  will  never  be 
exposed  to  the  rigors  of  warfare.  Insofar 
as  reliability  and  maintainability  are  con- 
cerned, no  commercial  firm  worth  its  fiscal 
salt  would  buy  a flight  simulator  that  oper- 
ated "in  the  red."  Our  counterparts  in  the 
airlines  have  been  using  simulators  for  years, 
cost  effectively  and  training  effectively, 
but  without  Military  Standards.  Now  that's 
hard  for  a good  military  man  to  believe! 

BUY  COMMERCIAL 

The  idea  of  buying  commercial  is  not 
new,  but  it  has  made  little  headway  against 
the  inertia  (dare  we  say  the  simplicity  or 
even  lethargy?)  of  specifying  to  Military 
Standards.  As  early  as  1972,  Senator  Chiles' 
Commission  on  Government  Procurement  pro- 
mised strong,  continued  Congressional  sup- 
port for  revised  procurement  policies.  To 
put  teeth  in  that  promise,  the  Commission 
also  proposed  the  establishment  of  the  Office 
of  Federal  Procurement  Policy  (OFPP)  within 
the  Executive  Branch.  The  OFPP  has  recently 
bared  those  teeth  in  memoranda  t~  the  Secre- 
tary of  Defense,  and  other  agencies,  recom- 
mending (the  Government's  euphemism  for  "do 
it")  that,  where  and  when  possible,  the  Agen- 
cies buy  commercially  available,  off-the- 
shelf  products  and,  further,  that  commercial 
distribution  channels  be  used  to  supply  these 
products  tb  their  users. 

More  to  the  present  point,  however,  is 
the  OFPP  guidance  for  developing  new  specifi- 
cation standards  and  practices.  The  guidance 
is  an  ampl ification  of  recommendation  made  by 
the  Commission  on  Government  Procurement  that 
the  Government  specify  performance  rather 
than  design. 

The  unfortunate  habit  of  writing  "tight" 
(read  design)  specifications  heavily  laced 
with  Military  Standards  has  kept  many  quali- 
fied bidders  out  of  the  marketplace,  and  has 
often  placed  the  Government  in  the  position  of 
having  to  buy  sole  source,  a position  frowned 
upon  by  the  GAO  and  the  Armed  Services  Procure- 
ment Regulations  (ASPR).  In  point  of  fact, 
both  ASPR  and  logic  state  that  competition  is 
the  name  of  the  game,  and  it  seems  clear  that 
if  we  were  to  write  performance  specifications 
allowing  industry  the  latitude  of  innovation, 
we  would  see  considerably  greater  response  to 
,r  requests  for  training  effective  proposals. 

DESIGN  TO  COST 

Now  for  another  not-so-new  idea-  design- 
- t cost  It  would  be  naive  not  to  admit 

• ■ years  industry  has  been  building  to 


cost.  However,  neither  side  has  been  willing 
to  admit  that  both  were,  equally  aware  of  bud- 
geted and  target  dollars.  It  is  reasonable 
to  presume  that  the  candor  of  requests  for 
design-to-cost  proposals  tied  to  performance 
specifications  would  generate  a more  competi- 
tive market  and,  as  a result,  a better  pro- 
duct for  the  money.  In  addition  to  the  ob- 
vious benefits,  design-to-cost  packages  will 
enable  the  contracting  officer  to  award  on 
the  basis  of  quality  rather  than  low  bid. 

Admittedly,  the  new  look  in  trainer  ac- 
quisition will  require  changes  in  legislation, 
policy,  and  administration,  but  we  already 
enjoy  the  full  support  of  the  Legislative  and 
Executive  branches  of  Government,  and  the  need 
for  quality  training  has  never  been  greater. 

TOTAL  EFFECTIVENESS 

Our  job  is  to  provide  the  most  effective 
device  at  the  lowest  cost.  Heretofore  we  have 
been,  for  the  reasons  stated,  myopically  pre- 
occupied with  cost  effectiveness.  Myopic  to 
the  extent  we  have  evaluated  the  cost  trade- 
offs we  can  effect  through  simulation  to  the 
nearly  complete  exclusion  of  attempting  to 
optimize  the  true  cost  of  a given  device.  We 
have  become  enamored  with  the  flight  hour  sub- 
sitution  that  we  can  effect  to  the  exclusion 
of  all  else. 

We  have  nearly  forgotten  our  primary  need 
for  effective  training.  After  all,  a training 
device  is  nothing  if  it  doesn't  train. 

TRAINING  EFFECTIVENESS 

Cost  effectiveness  discussion^  riease  us; 
we  are  comfortable  in  them.  One  reason  is  that 
we  have  an  accepted  unit  of  cost  effectiveness 
measurement  - the  dollar.  But  what  is  the  unit 
of  measurement  for  training  effectiveness? 
Learning  objective?  The  marginal  propensity 
for  knowledge?  Training  transfer  - that's 
it!  But  how  do  you  measure  training  effective- 
ness? The  unit  of  measurement  doesn't  yet  ex- 
ist. The  Educational  Specialists  are  franti- 
cally in  search  of  such  a unit. 

CREATE  AN  ENVIRONMENT 

But  we  in  the  training  business  do  have  a 
ready  made  unit  we  can  intuitively  relate  to 
until  the  Educational  Specialists  fill  the 
analytical  gap.  The  unit  to  which  we  refer  is 
user  acceptance.  Where  it  is  high,  utiliza- 
tion of  installed  devices  is  high.  Where  it  is 
low,  the  inverse  is  true.  Where  devices  have 
been  acquired  to  create  a badly  needed  tactical 
environment,  utilization  has  soared.  Where 
devices  have  been  provided  solely  for  reduc- 
tion of  flight  hours,  unprodded  utilization 
has  been  nearly  zero. 
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One  of  the  most  illogical  premises  with 
which  we  deal  today  is  that  which  states  "an 
increase  in  fidelity,  complexity,  or  duplica- 
tion perfection  is  axiomatically  an  increase 
in  training  effectiveness."  The  inverse  is 
undoubtedly  true  in  many  cases  where  one 
seriously  evaluates  the  device's  true  contri- 
bution to  training  effectiveness. 

A NEW  BALL  GAME 

But  all  is  not  lost.  We  can  hold  on  to 
our  present  advantage  if  we  take  the  time  to 
develop  a plan  for  the  future.  In  a sense, 
our  situation  is  similar  to  that  of  a strug- 
gling football  team  at  half  time.  We  have 
weathered  the  first  half,  the  period  of  buy- 
ing for  the  sake  of  savings.  Now  we  must 
revise  our  game  plan.  We  need  a plan  that 
more  effectively  recognizes  training  objec- 
tives realizing  that  the  only  thing  that  can 
sustain  us  in  the  long  run  is  a program  that 
is  built  upon  a continual  awareness  of  our 
training  needs. 


The  challenge  is  open  to  Government  and 
industry  alike,  and  the  stakes  are  high.  If 
we  do  not  work  together  in  this  new  direction, 
our  failure  to  do  so  can  only  be  measured  in 
terms  of  reduced  fleet  readiness;  an  unac- 
ceptable result. 


We  pledge  our  efforts  in  determining  and 
seeking  procurements  to  fulfill  only  those 
operational  requirements  that  are  effective 
from  a training  standpoint.  We  enjoin  you, 
contractor  and  military  procurement  personnel 
alike,  to  pledge  your  efforts,  expend  your 
energy,  and  use  your  imagination  in  affecting 
quantum  improvements  in  cost  effectiveness. 


The  problem  is  summarized  in  the  retiring 
remark  of  an  old  but  very  wise  procurement  of- 
ficer who  said:  "a  trainer  isn't  a trainer 

if  it  can't  train  --  but  a trainer  isn't  if 
I can't  buy  it. !" 
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USING  CAI  TO  MEASURE  TEAM  READINESS 


NORMAN  COPPERMAN 
Honeywell 

Marine  Systems  Division 
and 

PAUL  ASA  DORIAN 
FLEASWTRACENPAC 


COMNA VSURFPACINST  C3590.  1 requires 
for  all  ASW  platforms  a four-phase  ASW  training 
and  readiness  improvement  program,  with  the 
first  three  phases  conducted  at  FLEASWTRACEN- 
PAC and  the  fourth  phase  conducted  at  sea.  This 
instruction  requires  that  "All  training  conducted 
...  be  evaluated  and  assigned  a numerical  grade. 
In  addition,  the  ship  will  be  provided  copies  of  de- 
tailed grading  sheets  on  each  team  member.  " 

Phase  I,  Basic  Attack  Team  Training,  con- 
sists of  two  days  for  ships  equipped  with 
AN/SQS-23  and  three  days  for  ships  equipped  with 
AN/SQS-26.  It  involves  classroom  instruction  in 
terminology,  plotting,  tracking,  classification, 
search,  localization,  and  single  ship  attack  pro- 
cedure. It  also  includes  trainer  problems  that 
begin  with  single  ship/single  target  and  progress 
to  dual  ship/ aircraft  coordinated  operations  (in- 
cluding CZ  and  LAMPS),  culminating  with  graded 
attack  exercises.  The  14A2  ASW  team  trainer  is 
the  backbone  of  the  training  program. 

Surface  Ship  ASW  Attack  Trainers  of  the 
I4A2  Series  are  essentially  shorebased  ASW  tac- 
tics training  systems  that  provide  full  simulation 
of  the  sensors,  tracking,  fire  control  and  weapon 
deployment  systems  normally  provided  on  ASW 
ships  of  the  fleet.  Device  14A2  simulates  the  en- 
tire ASW  tactical  situation,  from  submarine  tar- 
get acquisition,  through  tracking,  fire  control 
solution,  and  weapon  firing.  It  simulates  own 
ships,  targets,  and  weapon  ballistics,  using  a 
general-purpose  digital  computer  to  generate  and 
control  problems. 

The  present  method  for  measuring  perform- 
ance is  broken  down  with  the  following  weighting 
factors: 


ASW  ESCORT  QUALIFICATION  PROGRAM 
K-000-1070  (PHASE  I) 

Performance  Measuring  and  Reporting  Guide 


Position  Points 

Comm  and/ Evaluator  28 

Bridge  4 

Sonar  8 

Underwater  Battery  Plot  12 

CIC  (General/ Plotting)  12 

Air  Control  16 

Weapons  20 


Penalty  Factors  to  be  Applied  to  All  Final 
Evaluations 

-5  Simulated  weapons  launch  such  that  friendly 
unit(s)  are  endangered. 

-5  Failure  to  attack  submarine  prior  to  sub- 
marine attack  on  friendly  unit. 

-3  Improper  sound-powered  phone  procedures 
and  failure  of  controlling  station  in  main- 
taining proper  discipline. 

Overall  Grade 

This  program  is  designed  to  maintain  and, 
if  possible,  improve  fleet  readiness.  But  does 
it?  The  answer  is  a disquieting,  "We  don't 
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know. " The  reason  for  the  difficulty  in  determin- 
ing the  degree  of  fleet  readiness  in  ASW  operations 
is  that  the  validity  of  the  scoring  system  used  for 
measuring  team  performance  is  questionable. 

At  first  glance,  the  indisputable  scoring 
tends  to  obscure  the  fact  that  some  aspects  of 
performance  are  so  loosely  defined  that  they  are 
subject  to  varying  levels  of  interpretation  by  dif- 
ferent instructors  at  different  14A2  sites.  Cur- 
rently in  the  14A2  trainers,  adequacy  of  perform- 
ance is  determined  by  the  instructor,  based  on 
his  individual  judgment,  because  objective  meas- 
urements, methods,  or  criteria  for  the  ASW  team 
are  nonexistent.  In  Figure  1,  for  example,  it  is 
hard  to  imagine  that  all  instructors  on  East  and 
West  coasts  will  agree  on  the  proper  forming  up 
of  the  SAU  and  the  correct  spacing  of  units,  par- 
ticularly since  there  are  no  standard  scenarios. 

A valid  scoring  system  requires  the  use  of 
objective  methods  to  evaluate  training.  However, 
the  problem  of  developing  objective  performance 
criteria  for  evaluating  surface  ASW  team  training 
is  an  elusive  one.  The  problem  is  compounded  by 
the  lack  of  specific  training  objectives  that  can  be 
readily  converted  into  measurable  criteria.  The 
remainder  of  this  paper  addresses  certain  tech- 
niques explored  for  establishing  an  objective  basis 
for  ASW  team  evaluation. 

PROBLEM 

The  development  of  a system  based  on  ob- 
jective criteria  for  training  and  evaluating  ASW 
team  performance  represents  a sizable  undertak- 
ing. The  first  step  has  been  one  of  data  col- 
lection and  analysis.  The  goal  for  this  phase  was 
to  determine  what  computer  aids  could  be  develop- 
ed to  help  make  more  objective  the  evaluation  of 
team  performance.  A ground  rule  during  this 
phase  was  that  training  practices  were  not  to  be 
considered  as  variables  but  were  to  be  defined 
as  presently  employed. 

APPROACH 

Computer  representations  of  several  train- 
ing sessions  were  "captured  t"  and  various  comput- 
er techniques  for  evaluating  team  performance 
were  explored.  The  I tA 2 computer  program  was 
modified  to  dump  to  magnetic  tape,  on  a second- 
by-second  basis,  the  computer  system  parameter 
tables.  With  this  modified  program,  tapes  con- 
taining all  relevant  variables  that  would  be  avail- 
able at  'he  time  of  training  were  generated  at 
FLEASWTRACENPAC  from  normal  training  ses- 
sions. In  addition,  a system  of  programs  was 
developed  for  scanning  the  tapes  for  particular 
performance  variables  and  for  providing  output 
in  suitable  formats. 

This  approach  permitted  an  empirical  de- 
termination of  an  analysis  technique  tailored  to 
the  special  requirements  of  ASW  team  training. 


Computer  assessment  of  performance  must 
rely  on  performance  criteria  that  are  specified  in 
terms  pf  computer  representations.  If  the  as- 
sessment is  to  be  meaningful,  these  criteria 
should  be  valid  over  a reasonable  range  of  pos- 
sible training  problem  scenarios.  Under  prevail- 
ing training  practices,  it  would  appear  virtually 
impossible  to  derive  a set  of  such  criteria. 

As  Device  14A2  is  currently  used,  problems 
are  specified  in  terms  of  the  vehicles  involved  and 
their  starting  configurations.  As  a training  mis- 
sion progresses,  problem  variables  change  in  a 
freely  developing  manner.  Performance  criteria 
derived  on  the  basis  of  such  a problem  specifica- 
tion and  valid  over  the  entire  range  of  possible 
outcomes  would  be  very  difficult  to  imagine. 

Even  if  problems  were  completely  "canned"  ex- 
cept for  own  ship  parameters,  the  derivation  of 
criteria  would  remain  indeterminable.  For  every 
such  canned  problem,  there  are  a number  of 
possible  tactics,  and  these  tactics  can  define 
contrasting  sequences  of  tasks  for  team  members. 
With  any  conceivable  set  of  criteria,  significant 
errors  in  evaluation  could  result.  Any  practical 
computerized  scoring  system  would  not  be  able 
to  adjust  to  the  tactics  used. 

Although  computer  assessment  appeared 
infeasible,  i;  still  seemed  reasonable  to  assume 
that  the  computer  could  be  a valuable  tool  in  the 
assessment  process.  Accordingly,  the  problem 
chosen  for  study  was  the  development  of  a display 
of  information  critical  to  a broad  range  of  train- 
ing problem  developments.  The  amount  of  in- 
formation had  to  be  limited  so  that  the  evaluator 
would  not  be  overwhelmed  with  it,  and  it  had  to  be 
displayed  so  that  performance  level  could  be  as- 
sessed with  a minimum  of  analytical  effort. 

Currently,  team  member  performance 
evaluation  in  Device  14A2  occurs  in  the  form  of 
a subjective  post-problem  critique.  The  analysis 
relies  on  the  questionable  analytical  ability  of 
team  members  in  that,  as  part  of  the  analysis, 
they  are  called  upon  to  search  their  memories  to 
try  to  report  the  causes  of  failures.  It  appeared 
that  there  was  much  that  could  be  done  to  aid 
this  process. 

The  only  system-generated  records  of  team 
performance  are  in  the  form  of  slides,  which  are 
normally  lost  after  training.  Clearly,  there  is 
room  for  improvement  in  the  capability  of  the 
system  to  generate  useful  records.  The  com- 
puter system,  in  addition  to  its  current  functions, 
could  be  used  to  collect  relevant  performance  data 
in  real-time  and  reduce  it  to  meaningful  statistics 
immediately  following  termination  of  a problem. 
The  results  could  be  in  the  training  officer's 
hands  in  time  for  his  critical  review.  This 
feature  would  enable  the  training  officer  to  derive 
an  objective  analysis  of  failures,  independent  of 
the  biased  views  of  the  participants. 
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COMMAND/EVALUATOR 


AVAIL. 

EARNED 

10 

10 

10 

20 

8 

6 

8 

6 

6 

8 

8 

6 

8 

5 

5 

7 

5 

8 

6 

COMMAND/EVALUATOR  - GENERAL 


Was  SAU  Commander  aware  of  tactical  employment  of  all  units 
under  his  command? 

Was  a low  noise  level  maintained  in  CIC  ? 

Was  classification  a continuous  process  by  SAU  Commander? 

Were  four  weapon  attacks  successful  ? (An  urgent  attack  does  not 
need  to  be  a hit  in  order  to  be  considered  successful. ) 


APPROACH  PHASE 


Was  Datum  properly  disseminated  to  all  units  ? 

Was  SAU  properly  formed  up,  and  was  spacing  correct,  considering 
predicted  sonar  range? 

Was  SAU  search  front  properly  reordered  and  was  approach  to 
Datum  proper  for  tactical  situation? 

Were  Cone  of  Courses,  Intercept  Course,  and  time  to  enter  TDA 
compared  and  concurred  with  Assist  Ship? 

Were  appropriate  countermeasures  executed  during  Approach 
Phase  ? 

Were  Plans  Red  and  Black  and  Weapons  Policy  passed  to  Assist 
Ship? 

Was  controlling  station  properly  kept  informed? 

Were  aircraft  plots  evaluated  to  determine  target  course  and  speed? 

Was  time  to  enter  TDA  updated  with  latest  contact  information  and 
new  Intercept  Course  executed  accordingly? 

SWAP  SITREP  requested  and  obtained  prior  to  execution  of  SWAP. 

SWAP  SITREP  disseminated  to  command,  with  State,  Weapons, 
Contact  status,  and  Datum  information  included. 

SWAP  executed  in  a timely  manner. 

SAU  advised  whe  SAC  is  assumed. 

Zig-Zag  Plan  executed  prior  to  entering  TDA. 

Appropriate  Material  Countermeasures  prepped  or  executed. 


Figure  1.  ASW  Escort  Qualification  Program  Evaluation  Form 
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PROGRESS 

Two  types  of  materials  have  been  developed: 
a graphical  representation  of  training  problem 
events,  and  a listing  of  numerical  information 
associated  with  critical  occurrences.  From  a 
ploi,  the  tactical  situation  and  the  performance  of 
all  subteams  except  for  CIC  can  readily  be  deter- 
mined. This  graph  is  similar  to  the  plot  cur- 
rently produced  in  slide  form  by  Device  14A2, 
but  it  contains  considerably  more  information, 
and  it  is  presented  in  hard-copy  form. 

Figure  2 shows  a plot  of  a simple  exercise: 
single  ship  (own  ship)  with  one  submarine.  This 
plot  was  produced  from  a" sample  of  the  collected 
computer  data.  The  black  and  red  trackings  are 
the  tracks  of  own  ship  and  enemy  submarine, 
respectively.  Unlike  the  Device  14A2  slides, 
here  time  indices  appear  on  the  tracks  for  use  in 
determining  the  relative  position  of  ships  in  any 
problem  and  at  any  time.  With  very  little  prac- 
tice, progression  of  the  tactical  situation  can 
also  be  determined.  On  each  track,  positional 
time  is  indicated  in  minutes;  the  time  between 
each  subunit  tick  is  10  seconds.  The  symbol  X 
on  a track  indicates  the  firing  of  a torpedo.  A 
dashed  line  connecting  the  "fire"  symbol  to  a 
small  circular  track  depicts  the  flight  path  of  an 
ASROC-launched  torpedo.  The  circular  track 
depicts  the  helical  search  pattern  of  a Mk  46  tor- 
pedo. The  time-tagged  torpedo  tracks  are  over- 
the-side  shots  by  own  ship  and  enemy  submarine. 

The  green  track  is  the  tracing  of  the  sonar 
operator's  cursor.  This  track  represents,  when 
systematically  in  the  vicinity  of  the  submarine, 
where  in  the  ocean  the  sonar  operator  was  speci- 
fying the  submarine  to  be.  As  can  be  seen  in 
Figure  2,  there  appears  to  be  a constant  range 
error  in  sonar  target  tracking,  which  could  re- 
flect a deficiency  in  the  sonar  operator's  per- 
formance or  perhaps  result  from  uncalibrated 
equipment.  The  tracing  of  sonar  tracking  is  not 
currently  available  on  Device  14A2. 

The  blue,  sawtooth-like  markings  that  run 
along-side  the  red  submarine  tracing  indicate 
computed  course  and  speed.  They  are  shown  at 
all  problem  times,  even  though  at  various  times 
they  may  be  meaningless,  such  as  computed 
values  before  initial  contact  has  been  made.  At 
each  intersection  of  a red  time  tick  and  the  red 
track  (at  the  submarine's  position  every  10  sec- 
onds), a blue  vector  is  drawn,  showing  a pre- 
dicted position  of  the  submarine  at  an  elapsed 
time  of  10  seconds,  based  on  the  actual  position 
of  the  submarine  and  its  computed  course  and 
speed.  Vector  direction  indicates  computed 
course,  and  vector  length  indicates  computed 
speed.  At  the  end  of  the  vector  is  a time  tick. 
When  computed  course  and  speed  are  perfect, 
the  blue  and  red  tracks  are  completely  superim- 
posed. Figures  3 and  4 are  further  examples  of 
training  problem  plots. 


Figure  5 is  a numerical  printout  designed  by 
FLEASWTRACENPAC  training  personnel.  The 
printout  lists  training  problem  information  at 
selected  times.  A line  of  printout  is  generated 
when  any  of  the  following  conditions  is  true: 

1.  (S)  Sonar  pulse  length  is  changed. 

2.  (E)  An  event  symbol  is  requested  by  an 

instructor. 

3.  (F)  A weapon  is  fired  by  own  ship. 

4.  (W)  An  ASROC-launched  torpedo  enters 

the  water. 

The  information  shown  here  under  runs  1,  2,  and 
3 was  derived  from  the  data  used  to  generate  the 
three  plots  shown  in  Figures  2,  3,  and  4, 
respectively. 

SIGNIFICANCE  OF  RESULTS 

With  the  exception  of  the  CIC,  the  plots 
present,  in  comparative  form,  significant  actions 
of  all  subteams.  Shown  together  in  a single 
representation,  it  should  be  possible  to  assess 
how  well  individual  subteams  performed  and, 
more  importantly,  how  well  coordinated  their 
actions  were.  With  the  addition  of  CIC  plotting 
information,  the  performance  of  all  subteams 
would  be  represented,  and  an  evaluator  could 
readily  determine  which,  if  any,  of  the  subteams 
was  responsible  for  failures  in  the  team's  exe- 
cution of  its  tactic. 

Immediate  availability  of  the  plots  and 
training  statics  following  a training  session 
would  free  the  instructor  from  many  of  his 
record-keeping  tasks  and  allow  him  to  evaluate 
team  communications  more  thoroughly. 

A plot  serves  as  the  basis  of  a permanent 
record  of  team  performance,  and  a library  of 
plots  represents  a fleet  performance  data  base. 
Such  records  have  potential  value  in  several 
respects: 

1.  An  incontrovertible  evidence  of  team 
failures  (for  use  when  a ship  challenges 
its  performance  grade). 

2.  A source  of  performance  statistics, 
permitting  an  objective  description  of 
fleet  readiness. 

3.  The  means  for  determining  the  effec- 
tiveness of  training  methods  and 
training  devices. 

4.  A basis  for  evaluating  ASW  doctrine. 
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Figure  5.  Training  Problem  Printout 


PHASE  n PLAN 


The  next  phase  of  the  program  will  include 
development  of  a variety  of  computer-based  prob- 
lem scenarios.  These  scenarios  will  provide  a 
training  option  that  should  increase  training  prob- 
lem standardization  by  placing  target  maneuvers 
under  computer  control. 

At  present,  target  maneuvers  are  under 
instructor  control.  Maneuvers  employed  in  train- 
ing reflect  the  instructors'  differing  concepts  of 
submarine  tactics.  In  addition,  instructors  use 
their  individual  judgment  in  determining  how  a 


submarine's  maneuvers  should  be  adjusted  to 
achieve  a chosen  level  of  problem  difficulty.  Un- 
der these  conditions,  it  is  virtually  impossible  to 
derive  performance  norms  to  cover  the  range  of 
possible  tactical  developments. 

Standardized  scenarios  should  permit  a valid 
comparison  of  the  performance  of  ASW  teams  of 
different  ships,  trained  at  different  times,  at 
different  sites,  and  by  different  instructor  staffs. 
The  goal  of  this  next  phase  will  be  to  develop 
scenarios  that  are  graded  with  respect  to  prob- 
lem difficulty  and  that  cover  a representative 
range  of  tactical  possibilities. 
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AN  APPROACH  TO  STIMULATION  OF  OCEAN 
MULTIPATH  PHENOMENA  FOR  SONAR  TRAINING  DEVICES 


MICHAEL  F.  STURM  and  IRWIN  S.  FROST 
Honeywell 

Marine  Systems  Division 


To  meet  the  increasing  threat  imposed  by 
well- equipped,  modern  submarines,  active  and 
passive  sonar  systems  with  highly  sophisticated 
signal  processing  and  display  subsystems  must  be 
developed.  Training  of  sonar  operators  to  use 
these  systems  effectively  requires  simulation  of 
actual  operations  whenever  possible.  This  can  be 
achieved  by  providing  a training  system  that: 

(1)  simulates  operational  equipment,  or  (2)  stimu- 
lates the  actual  tactical  hardware.  The  trainer 
must  present  the  trainee  with  the  same  operating 
situation  that  he  wall  encounter  in  real-world  com- 
bat conditions,  including  the  ocean  acoustic  model. 

Training  system  models  of  the  ocean  acous- 
tic environment  must  be  adequate  to  represent  the 
new  sonar  systems,  with  their  increased  sophis- 
tication and  improved  signal  processing  capabil- 
ities. Consequently,  new  and  more  complete 
acoustic  environmental  models,  which  can  run  in 
real-time,  must  be  developed  for  both  active  and 
passive  mode  sonar  systems  trainers.  To  stay 
within  Navy  budgets,  however,  innovative,  cost- 
effective  concepts  and  designs  are  required  for 
the  models,  as  well  as  the  stimulation  hardware. 

The  amount  of  nontactical  hardware  required 
for  the  development  of  sonar  training  devices  us- 
ing stimulation  techniques  is  roughly  proportional 
to  the  number  of  acoustic  paths  that  mi:-r.  be  simu- 
lated. The  sounds  of  all  acoustic  paths  between  a 
source  and  a receiver  must  be  generated, 
frequency-shaped,  attenuated,  and  inverse- 
beamformed.  A reduction  in  the  total  number  of 
acoustic  paths  to  a subset  sufficient  for  stimula- 
tion, while  retaining  the  inherent  attributes  of  the 
ocean  multipath  phenomenon  and  sonar  capabil- 
ities, permits  effective  training  with  a minimum 
amount  of  nontactical  hardware. 

The  objective  of  this  paper  is  to  discuss  a 
concept  and  associated  analytic  processes  de- 
veloped for  use  in  sonar  training  devices  to  pro- 
vide high-fidelity  simulation  of  ocean  multipaths. 

It  is  shown  that  this  technique  can  be  used  to  rep- 
resent the  total  acoustic  field,  using  two  equivalent 
multipath  subsets. 

OCEAN  ACOUSTICS 

Accurate  representation  of  the  acoustic  field 
between  any  two  points  in  the  ocean  environment 
requires  the  consideration  and  accurate  represen- 
tation of  all  major  contributing  acoustic  paths  be- 
tween these  points.  As  the  whole  Is  the  sum  of  all 
its  individual  parts,  the  total  acoustic  field  is  the 
sum  of  the  individual  multipath  contributions.  It 
is  well-known  that,  between  any  two  spatial  loca- 
tions, acoustic  energy  can  propagate  along  many 


individual  paths,  the  nature  of  which  is  solely  de- 
pendent upon  the  sound  velocity  structure  of  the 
medium  itself.  Typical  examples  of  this  phenom- 
enon are  illustrated  in  the  acoustic  ray  diagrams 
and  associated  sound  velocity  profile  structures 
shown  in  Figures  1 and  2.  The  individual  ray 
paths  joining  any  two  points  in  the  medium  are 
referred  to  as  eigenrays.  It  is  the  summation  of 
the  acoustic  characteristics  of  each  of  the  indi- 
vidual eigenray  intensities  that  determines  the 
field  intensity  at  a given  location  in  the  ocean. 

The  summation  of  these  eigenrays  produces 
such  well-known  ocean  characteristics  as  multi- 
path  interference  regions  and  convergence  zones. 
Figure  3 illustrates  the  acoustic  field  enhance- 
ment, a decrease  in  propagation  loss,  due  to  the 
convergence  of  the  individual  eigenrays. 

Since  it  is  not  within  the  scope  of  this  paper 
to  delineate  acoustic  ray  theory,  a less  rigorous, 
yet  illustrative,  example  will  be  used  to  intro- 
duce the  subject  simulation  problem  and  solution. 

Consider  the  simplified  two-boundary  image 
model  geometry  shown  in  Figure  4.  For  illustra- 
tive purposes,  multipaths  between  the  source  and 
receiver  are  assumed  to  be  straight  lines;  six 
individual  paths  are  shown.  The  total  acoustic 
field  (assuming  unity  surface  reflection  and  con- 
stant sound  speed)  at  any  spatial  coordinate  is 
represented  by  the  coherent  sum  of  each  individ- 
ual multipath  and  is  analytically  given  by, 
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Figure  3.  Typical  Propagation  Loss  Plot  Illustrating  Convergence  Zone 


Information  associated  with  each  of  these 
multipaths  includes  intensity  (spreading  and  ab- 
sorption loss,  bottom  and  surface  loss),  phase, 
arrival  angle,  and  travel  time  or  differential 
travel  time.  In  this  example,  path  length  and  time 
are  functionally  equivalent,  since  a constant  speed 
of  sound  is  assumed.  In  the  more  rigorous  sense, 
this  is  not  true,  since  the  actual  acoustic  energy 
travelB  along  ray  paths  in  a medium  at  a contin- 
uously varying  sound  speed. 

Figure  5 illustrates  a typical  propagation 
loss  output  from  the  aforementioned  Image  model. 
To  provide  high-fidelity  simulation  of  ocean  phe- 
nomena commensurate  with  the  operational  capa- 
bilities of  the  tactical  hardware,  the  various 


attributes  of  multipath  propagation,  as  summarized 
above,  must  be  considered  for  effective  sonar 
training.  If  the  tactical  hardware  is  sensitive  to 
the  effects  of  multipath  phenomena,  these  effects 
must  be  simulated  to  a fidelity  equivalent  to  the 
hardware  capabilities. 

APPLICATION 

Generlcally  speaking,  sonar  or  nonsonar 
training  devices  can  be  divided  into  two  basic 
categories:  (1)  full  simulators,  not  employing 
tactical  hardware;  In  these  devices,  visual  and 
aural  trainee  presentations  are  almost  completely 
under  the  control  of  mathematical  processes;  and 
(2)  stimulators,  in  which  synthetic  signals  are 
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Figure  4.  Two- Boundary  Image  Model  Geometry 
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injected  at  some  point  into  the  tactical  hardware 
under  the  control  of  mathematical  processes. 

The  process  of  achieving  a cost-effective 
trainer  design  often  includes  development  of 
synthetic  signals  that  do  not  completely  map  real- 
world  phenomena  but  are  "acceptable  for  training." 
For  a sonar  training  system  using  stimulation 
techniques,  the  synthetic  signals  must  be  more 
nearly  representative  of  the  real-world  acoustic 
signals  if  the  tactical  equipment  is  to  be  "fooled 
The  further  the  point  of  injection  is  moved  to  the 
front  end  of  the  system,  the  greater  the  fidelity 
required  of  these  synthetic  acoustic  signals. 

In  a sonar  trainer  employing  full  simulation 
or  stimulation  with  injection  after  the  beamformer, 
effects  of  multipath  phenomena  can  be  functionally 
simulated  and  represented  in  a purely  analytic 
manner,  irrespective  of  the  complexity  of  the 
mathematical  processes  required,  as  shown  in 
Figure  G. 

A sonar  trainer  that  uses  synthetic  signal 
injection  points  at  either  the  input  or  output  of  the 
array  sensor  pre-amps  and  depends  on  the  sensi- 
tivity of  the  system  to  multipath  signals  requires 
the  use  of  an  inverse  beamformer  and  associated 
signal  generator  for  each  target  multipath  combi- 
nation, as  illustrated  in  Figure  7.  The  inverse 
beamformers  are  required  to  time-orient  the 
synthetic  signals  in  both  the  vertical  and  azimuthal 
planes  so  that  the  spatial  geometry  of  the  array 
sensors  is  accounted  for  prior  to  signal  injection 
into  the  tactical  hardware  beamformers. 

ALGORITHM  OVERVIEW 

The  primary  computer  program  used  to  de- 
velop the  concepts  presented  in  this  paper  is  the 
Navy  Interim  Surface  Ship  Model  (N1SSM)  II  (Ref. 

1).  NISSM  was  developed  by  Dr.  Henry  Weinberg 
of  the  Naval  Underwater  Systems  Center  at  New 


London,  Connecticut,  to  predict  the  performance 
of  sonar  systems.  The  program  provides,  as  a 
function  of  ocean  and  system  parameters,  acous- 
tic ray  paths,  propagation  loss  predictions, 
boundary  and  volume  reverberation,  signal-to- 
noise  ratios,  and  probability  of  detection  outputs, 
using  continuous-gradient  ray-tracing  techniques. 
Model  assumptions  include  horizontal  ocean 
boundaries  and  a non- range-dependent  sound 
velocity  profile. 

The  algorithm  developed  for  selecting 
"bundled"  multipaths  is  currently  automated  on  a 
CDC  G600  as  part  of  an  in-house  NISSM  II  system. 
All  figures  presented  in  this  paper  were  created 
from  this  system. 

As  mentioned  earlier,  many  eigenrays  exist 
between  the  source  and  receiver.  Selection  of  the 
rav  path  and  associated  acoustic  information  for 
simulation  or  stimulation  is, therefore, of  prime 
importance.  If  selected  incorrectly,  many  of  the 
acoustic  phenomena  required  for  training  could  be 
inadvertently  omitted.  A typical  selection  cri- 
terior  that  is  sometimes  considered  is  to  provide 
the  acoustic  information  associated  with  the  two 
strongest  eigenrays.  This  is  a good  approxima- 
tion if  the  selected  eigenrays  are  sufficiently 
stronger  than  the  remaining  ones;  however,  it  is 
not  necessarily  a typical  situation.  Since  this  type 
of  selection  process  can  produce  Significant  er- 
rors In  certain  instances,  it  should  not  be  con- 
sidered the  optimal  approach.  For  example, 
consider  the  typical  plot  of  propagation  loss  versus 
range  presented  in  Figure  8.  The  solid  line  rep- 
resents the  coherent  summation  of  path  loss  along 
all  ray  paths.  The  line  broken  by  circles  repre- 
sents the  coherent  summation  of  the  path  loss 
using  the  two  strongest  rays.  More  significant 
than  the  errors  between  the  two  curves  is  the  fact 
that  they  do  not  have  the  same  general  shape. 
Therefore,  to  provide  effective  training  (again 
dependent  upon  the  tactical  hardware  capabilities) 
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Figure  7.  Target/Acoustic  Path  Function  Block  Diagram  for  Pie-Beamformer 
Stimulation  Implementation 
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Figure  8.  Comparison  of  NESM  vs  Two  Strongest  Rays  for  Mediterranean  Profile  - Summer 


a more  accurate,  cost-effective  representation  of 
the  acoustic  field  for  training  systems  must  be 
provided.  The  technique  provided  in  this  paper 
provides  the  required  cost-effective  fidelity,  using 
a process  that  permits  representation  of  the  total 
acoustic  field  by  two  equivalent  multipath  "bun- 
dles This  process  Is  described  In  the  remainder 
of  the  paper. 

BUNDLING  ALGORITHM 

The  subject  bundling  algorithm  uses  the  ef- 
fective coherent  propagation  loss,  which  is  cal- 
culated from: 


f = discrete  phase  angle  change  of  the  j 
1 eigenray  due  to  boundary  interactions 
(radians)  . 

Figure  9 Illustrates  the  bundling  algorithm  in 
flowchart  form.  The  algorithm  to  be  used  (ex- 
cept in  a caustic  region)  is  computed  as  follows: 

a.  Calculate  the  coherent  propagation  loss 
of  all  upward-going  eigenrays,  Ny. 

b.  Calculate  the  coherent  propagation  loss 
of  all  downward-going  eigenrays,  N^. 
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where 
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propagation  loss  for  the  j1*1  eigenray 
<dB) 


f ” frequency  (Hz) 

t * travel  time  for  the  J1*1  eigenray 
1 (seconds) 


c.  Select  the  strongest  upward  eigenray 
(j  * ju)  and  the  strongest  downward 

eigenray  (J  = j^),  combine  these  coher- 
ently, and  obtain  Nj,  using  Equation  1. 

d.  Select  the  strongest  upward  eigenray 
(j  *j)  such  that  f . 

‘ Ju 

e.  Coherently  combine  the  j'eigenray  path 
with  the  eigenray  path,  N2>  in  ac- 
cordance with  Equation  2. 


Figure  9.  Bundling  Algorithm  Flow  Chart 
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f.  Select  the  strongest  downward  elgenray 
(j  =T)  such  that??-  t f 


j 


j. 


g.  Coherently  combine  the  X eigenray  path 
with  the  j elgenray  path,  Ng,  In  ac- 
cordance with  Equation  3. 


h. 


Values  of  Nu  and  Nj  provide  the  propa- 
gation loss  values  roi 
paths. 


for  the  two  bundled 


Associated  acoustic  information  required  for 
simulation/stimulation  includes  angles  and  time 
of  arrival  and  discrete  phase  information,  which 
are  obtained  from  the  eigenray  paths  by  using  the 
following  criteria: 

(1)  If  Nj  is  closest  to  N^:  Paths  ju  and 

(2)  If  N2  is  closest  to  Neff:  Paths^and 

(3)  If  Ng  is  closest  to  Neff:  Paths  7 and  jy  . 

Figures  10  through  12  illustrate  the  outputs 
obtained  by  using  the  aforementioned  algorithms. 
The  parameters  for  Figure  8 and  10  are  identical, 
serving  to  illustrate  the  higher  fidelity  obtained  by 
using  the  bundling  algorithms,  as  opposed  to  se- 
lecting the  strongest  eigenrays. 

Figure  11  illustrates  typical  results  ob- 
tained in  well-defined  interference  regions.  Once 
again,  use  of  the  bundling  algorithms  provides  ex- 
cellent correlation  with  the  primary  N1SSM  II 
output. 

For  a caustic  region  typically  associated 
with  convergence  zones,  N1SSM  II  uses  a -90  de- 
gree phase  shift.  For  the  bundling  algorithm, 
once  Nu  and  Nj  are  calculated,  it  has  been  shown 
that  the  caustic  phase  shift  is  no  longer  required. 
Hence,  phase  information  provided  for  each 
bundled  path  is  represented  by  either  0 or  180 
degrees  to  account  for  the  surface  boundary  phase 
shifts.  To  use  the  aforementioned  algorithms 
within  a caustic  region,  the  following  step  should 
be  included  after  Step  b: 


bl.  Modify  all  phase  shifts  to  account  for 
surface  boundaries  only;  delete  the 
caustic  phase  shift  of  -90  degrees. 

Figure  13  Illustrates  the  application  of  the 
caustic  correction.  Once  again,  the  output  of  the 
N1SSM  II  model  and  that  provided  by  the  bundling 
algorithm  are  well  correlated. 
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Figure  10.  Comparison  of  NISSM  vs  Bundling  Algorithm  for  Mediterranean  Profile  - Summer 


Figure  11.  Comparison  of  ND3SM  vs  Bundling  Algorithm  for  Mediterranean  Profile  - Summer 
Multipath  Interference  Region 


Figure  12.  Comparison  of  NBSM  vs  Bundling  Algorithm  for  Mediterranean  Profile 


Figure  13.  Comparison  of  NISSM  vs  Bundling  Algorithm  for  Tyrrhenean  Sea  - Convergence  Zone 
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THE  SIMULATOR  INSTRUCTOR  - A READINESS  PROBLEM 


OR.  JOHN  P.  CHARLES 
Appl i-Mation , Inc. 


BACKGROUND 

The  increasing  pressure  to  extend  utiliza- 
tion of  simulator  training,  the  continuing  in- 
crease in  sophistication  of  weapon  systems, 
and  the  accelerating  application  of  advanced 
simulator  technology  are  creating  new  sets  of 
problems  for  the  simulator  training  system. 

Many  of  these  problems  are  centered  on  one  of 
the  key  sub-systems  - the  instructor.  Most  of 
these  problems  which  will  be  discussed  in  the 
following  paragraphs,  stem  from  the  failure 
to  treat  simulation  training  as  a system,  and 
in  particular,  the  failure  to  adequately  struc- 
ture the  role  of  the  instructor  and  to  design 
the  required  interface.  Thus,  while  wide- 
angle  visual  systems,  complex  motion  platforms 
and  sophisticated  CRT's  have  been  added,  the 
functions  and  interface  for  the  instructor 
have  been  largely  ignored  and  the  resultant 
instructor  console  becomes  a victim  of  design 
"fall-out."  To  complicate  the  problem,  the 
characteristics  of  the  instructor  pilot  and 
the  operational  training  concept  are  also 
changing  significantly. 

Feasible  solutions  which  have  been  ad- 
vanced, range  from  one  extreme  represented  by 
complete  automation  to  the  other  requiring  high- 
ly trained  and  specialized  simulator  instructors. 
Neither  is  currently  implementabte  or  accept- 
able. Yet,  little  other  effort  has  been  ex- 
pended to  resolve  the  problem. 

The  dilemma  created  has  parallels  in  other 
fields  where  full  scale  application  of  technol- 
ogy, especially  computer  technology  has  oc- 
curred with  almost  equivalent  disregard  for  the 
requirements  of  the  human  decision  maker.  Auto- 
mated materials  handling  presents  an  excellent 
example.  One  airline,  for  example,  after  in- 
stalling a sophisticated  automated  cargo  hand- 
ling system,  was  forced  to  fall  back  and  re- 
design the  system  - "We  had  to  go  back  to  the 
time  when  man,  not  tf\e  machine,  was  the  boss 
of  cargo  handling."  1 Perhaps  the  time  has 
also  arrived  to  return  to  the  concept  of  the 
instructor  as  the  "boss"  of  simulator  training. 

INSTRUCTOR  PILOT  STUDY 

Recognizing  that  major  changes  were  oc- 
curring in  simulator  design  and  that  instruc- 
tor problems  were  arising,  the  Human  Factors 
Laboratory  of  the  Naval  Training  Equipment 
Center  undertook  a study  of  the  instructor 
pilot's  role  in  simulator  training  with  the. 
ultimate  goal  of  defining  and  designing  the 
required  instructor  consoles.  The  effort 
included  a survey  of  the  current  role  of  the 


instructor  pilot  (IP)  in  readiness  training, 
an  analysis  of  instructor  functions  and  the 
development  of  instructor  console  design 
concepts  including  the  feasibility  of  mod- 
ularized and  standardized  consoles. 

The  study  began  with  an  analysis  of  a 
generic  simulator  training  system.  Figure 
1 illustrates  the  major  subsystems  con- 
sidered, i.e.,  the  student,  the  simulator, 
the  syllabus,  and  the  instructor.  The  pre- 
liminary analyses  were  directed  to  identi- 
fying the  interface  requirements  for  the 
instructor  subsystem  and  to  developing  a 
structure  for  the  collection  of  data  during 
the  survey  of  simulator  training. 

In  brief,  the  results2  showed  that  each 
of  the  training  subsystems  generate  broad 
demands  on  the  instructor  subsystem. 

• The  student,  e.g.,  requires  briefing, 
monitoring,  demonstration  feedback,  evalua- 
tion, and  debriefing. 

• The  syllabus  requires  implementation 
including  identification  and  selection  of 
appropriate  mission  and  performance  cri- 
teria as  well  as  structuring  of  simulation 
parameters  and  control  points. 

• The  simulator  requires  initialization 
and  operation,  and  additional  simulation 
support  such  as  air-traffic  control  and 
ground  crew  functions. 

Thus,  the  instructor  subsystem  provides 
the  integrating  function  for  the  entire 
training  system  as  well  as  providing  miss- 
ing simulation  models  and  ancillary  func- 
tions not  performed  by  the  other  sub- 
systems. As  Caro  pointed  out,  "A  training 
program  is  the  manner  in  which  the  well- 
qualified  instructor  uses  the  appropriately 
designed  simulator  to  establish  the  clearly 
defined  course  content  within  the  skills 
repertoire  of  the  trainee."1 

The  characteristics  of  existing  sub- 
systems were  surveyed.  The  following 
paragraphs  summarize  the  results. 

SYLLABUS  PROBLEMS 

The  typical  simulator  flight  training 
syllabus  is  based  on  or  "borrowed"  from  the 
flight  syllabus.  Thus,  it  rarely  identifies 
or  structures  in  the  required  detail  those 
parameters  and  functions  which  are  not  con- 
trollable in  flight.  This  is  not  a trivial 
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• radio-navigation  aid  variables  such  as 
type,  frequency,  identification 

• visual  variables  such  as  ceiling,  vis- 
ibility, lighting 

• malfunctions  and  emergencies  onset, 
development  and  resolution 

• simulator  operating  controls  such  as 
motion  actuation,  visual  actuation, 
start,  reset 

• training  controls  such  as  freeze,  demon- 
strate record,  replay 

• vehicle  configuration  including  fuel, 
stores 

• operational  environment  including  tar- 
gets and  characteristics,  jammers 

Thus,  a modern  simulator  will  contain  hun- 
dreds of  parameters  and  controls  which  must 
be  set.  While  some  automation  exists,  the 
impact  on  the  instructor  is  not  correspon- 
dingly reduced  since  he  must  at  least  verify 
the  configuration.  The  burden  on  the  instruc- 
tor is  obviously  sizeable  when  the  syllabus 
does  not  identify  or  structure  the  flight 
training  envelope  much  less  the  mission  sce- 
nario and  enemy  tactics. 

INSTRUCTOR  CONSOLE  PROBLEMS 

The  instructor  consoles  in  digital  opera- 
tional trainers  can  be  grouped  into  two  types 
based  on  vintage.  The  first  generation  trainer 
consoles  typified  by  the  F-4J  trainer  (2F88) 
can  be  described  in  terms  of: 

• "wraparound"  console 

• "repeater"  cockpit  instrument  panel 

• displays  and  controls  arranged  by  air- 
craft system 

The  consoles  were  reasonably  well  human 
engineered  within  the  constraints  involved. 

In  general,  they  were  designed  for  a two-  to 
three-man  operation,  at  least  one  of  whom  is 
a simulator  technician  well  qualified  in  manip- 
ulation of  the  complex  consoles.  Instructor 
personnel  - pilots  and  naval  flight  officers  - 
"boss"  the  training  evolution. 


The  second  generation  consoles  such  as 
the  T-2J  trainer  (2F101)  are  typified  by  the 
extensive  use  of  CRT's  and  keyboards  for  all 
display  and  control  functions.  Mechanization 
has  generally  involved  wide  use  of  both  text 
and  pictorial  displays.  Unfortunately,  it 
appears  that  the  wide  capability  of  graphics 
displays  has  led  to  an  instructor  console 
design  philosophy  of  providing  a display  and 
a control  for  any  and  everything.  Thus,  any 
requirement  is  bound  to  be  satisfied.  Unfor- 
tunately, this  has  led  to  CRT's  with  hundreds 
of  pages  of  text,  displays,  and  indexes  re- 
sulting in  a distinct  probability  that  the 
instructor  will  fall  behind  the  training  evo- 
lution in  the  first  minutes  of  operation. 

INSTRUCTOR  PROBLEMS 

The  instructor  subsystem  in  operational 
trainers  presents  another  set  of  problems 
which  closes  the  loop  on  the  dilemma.  The 
study  revealed  that  the  role  of  the  instruc- 
tor and  the  supporting  or  assisting  role  of 
the  technician  varies  with  the  nature  of  the 
weapon  system  involved.  In  single  place  air- 
craft systems,  the  technician  generally  con- 
tributes significantly  since  the  trainer  is 
generally  a weapon  system  trainer.  The  sys- 
tem itself  is  complex,  and  no  other  instruc- 
tors are  involved.  In  multiple-crew  air- 
craft, the  role  appears  to  vary  with  mission 
of  the  system  and  the  role  of  the  pilot  in 
the  system. 

Where  the  pilot  is  almost  exclusively 
concerned  with  vehicle  control,  the  IP  can 
and  does  conduct  the  simulator  training  gener- 
ally on  operational  flight  trainers  (OFT)  with 
little  if  any  assistance.  Where  the  pilot  is 
intimately  involved  in  multi-crew  system  opera- 
tion, the  IP  functions  as  one  of  several  sim- 
ulator instructors  at  the  console(s)  with 
other  instructor(s)  and  technician(s)  assis- 
tance. Yet,  in  general,  the  instructor  con- 
soles are  designed  alike;  i.e.,  with  little 
consideration  for  the  instructor  team  com- 
position or  functions. 

Finally,  the  IP,  almost  without  exception, 
is  not  trained  in  simulator  operation  or  more 
importantly,  in  simulator  utilization  in  sup- 
port of  the  syllabus.  Very  limited  on-the-job 
training  is  typically  provided  the  IP. 

In  summary,  the  typical  modern  simulator 
training  system  in  operation  can  be  character- 
ized by: 

a.  inadequately  designed  syllabus 

b.  ineffective  interface  or  console 

c.  inadequately  trained  IP 


PROBLEM  SUMMARY 


The  problem  is  multifaceted  and  inter- 
acting. Although  unigue  simulator  syllabus 
development  can  and  should  be  undertaken,  it 
must  consider  the  interaction  with  the  instruc- 
tor and  the  console.  The  console  or  inter- 
face cannot  be  well  designed  until  the  char- 
acteristics of  the  IP  and  the  syllabus  and 
training  objectives  have  been  identified.  The 
instructor  cannot  be  trained  until  his  role 
is  defined.  Thus,  the  problem  is  a typical 
systems'  engineering  problem  and  as  Machol 
summarized,  "Because  the  problem  cannot  be 
adequately  formulated  until  it  is  well  under- 
stood, and  because  it  cannot  be  well  under- 
stood until  it  has  been  more  or  less  solved, 
the  two  are  inseparable. "3 


(2)  Automated  or  technician  sup- 
ported simulator  initialization  and  opera- 
tion. 

(3)  A general  standardized  sylla- 
bus defined  in  operational  terms  for  IP 
use. 

(4)  A detailed  translation  of  the 
syllabus  in  simulation  implementation  terms 
for  support  personnel  (or  software  mechaniza 
tion) . 

On  the  other  hand,  if  highly  qualified 
and  trained  simulator  instructors  are  to  be 
utilized,  a unique  and  sophisticated  instruc- 
tor console  could  be  implemented  with  little 
similarity  to  the  operational  cockpit. 


Thus,  the  requirement  is  for  the  inter- 
active analysis  and  trade  of  performance, 
constraints,  and  design  alternatives. 

INSTRUCTOR  DESIGN  CONSTRAINTS 

The  study  of  the  IP  in  simulator  training 
revealed  that,  at  least  in  present  training 
operations,  the  instructor  subsystem  could  be 
characterized  as  follows: 

a.  The  simulator  IP  is  a relatively  ju- 
nior flight  instructor  virtually  untrained  in 
simulator  operation  or  utilization. 

b.  The  console  is  designed  for  simula- 
tion variable  control  rather  than  for  training 
functions. 

c.  Definitive  simulation  training  sce- 
narios are  seldom  provided. 

d.  Briefing/debriefing  capability  is 
limited. 

e.  Standard  ation  of  training  is  mini- 
mal . 

The  picture  is  not  unlike  other  system 
designs  in  which  each  component  is  designed 
independently  to  different  criteria.  Yet, 
tradeoffs  and  effective  designs  are  feasible 
to  most  mixes  of  console  designs  and  instruc- 
tor characteristics.  For  example,  if  qualified 
flight  instructors  are  to  be  utilized  with 
minimal  training  in  simulator  operation,  then 
the  characteristics  of  the  console  and  sylla- 
bus might  be  stated  as  including: 

(1)  An  IP  console  with  maximum 
similarity  to  the  cockpit  to  minimize  IP  train- 
ing and  to  take  advantage  of  his  already 
achieved  skill  in  evaluating  system  perfor- 
mance. There  would  be  little  doubt  that  a 
qualified  pilot  could  evaluate  aircraft  condi- 
tion more  rapidly  and  meaningfully  from  a du- 
plicate of  cockpit  displays  than  he  could  from 
alphanumeric  readouts  on  a CRT. 


Alternative  designs  depending  primarily 
on  instructor  characteristics  are  clearly 
possible.  Unfortunately , the  typical  pre- 
sent-day simulator  incorporates  both  an  in- 
structor console  and  a syllabus  which  are 
not  designed  to  meet  instructor  requirements. 

OVERVIEW 

Effective  simulator  training  can  only  be 
achieved  by  considering  the  simulator  as 
part  of  a training  system  which  includes  the 
instructor,  the  syllabus,  and  the  student. 
Design  constraints,  especially  in  terms  of 
instructor  characteristics  must  be  iden- 
tified and  reflected  in  the  design  of  the 
instructor  consoles.  Similarly  these  con- 
straints, along  with  training  objectives 
must  be  utilized  in  the  development  of  the 
simulation  training  syllabus  if  efficient, 
effective,  and  standardized  training  is  to 
be  achieved. 

The  findings  reported  in  the  study  of 
the  IP's  role  in  simulator  training  lead  to 
the  conclusion  that  the  instructor  personnel, 
at  least  for  the  near  future,  will  be  oper- 
ational personnel;  i.e.,  operationally  qual- 
ified pilots  and  Naval  flight  officers. 
Furthermore,  minimal  training  in  simulator 
utilization  will  be  provided  for  many  rea- 
sons including: 

a.  Simulator  training  will  be  "collat- 
eral duty"  along  with  flight  instruction 
and  other  squadron  duties. 

b.  Transfer  and  rotation  of  personnel 
will  preclude  any  extended  training  program 
in  simulator  utilization. 

c.  Multiple  training  functions  of  the 
instructor;  i.e.,  academics,  flight,  and 
simulation  will  preclude  specialization  in 
simulator  utilization. 

Thus,  simulator  instructor  consoles 
should  be  designed  for  utilization  by 
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operational  personnel  with  a minimum  require- 
ment for  training  in  simulator  utilization 
and  a maximum  utilization  of  existing  IP 
skills  and  abilities.  These  design  objec- 
tives are  readily  achievable  with  existing 
design  technology  and  methodology. 
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ESTABLISHING  TRAINING  CRITERIA  ON  AN  ECONOMIC  BASIS 
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Calspan  Corporation 


INTRODUCTION 

The  Problem 

The  training  community  has  toiled  for  many 
years  in  an  attempt  to  establish  exactly 
what  is  meant  by  the  phrase  "training  effec- 
tiveness." Although  no  one  can  precisely 
define  it,  the  concept  of  effective  training 
is  of  interest  to  psychologists  and  managers 
alike.  The  question  is  continuously  being 
posed  by  managers:  "is  our  training  program 

cost-effective?"  Subsequently,  training 
psychologists  begin  expounding  on  the  multi- 
tude of  factors  that  comprise  the  known 
principles  of  learning  (reinforcement,  habit 
patterns,  immediate  feedback,  etc.).  At  the 
end  of  the  discussion,  the  managers  conclude 
that  they  know  (viscerally)  that  training  is 
necessary  but  they  recognize  that  the  psychol- 
ogists cannot  convince  each  other,  let  alone 
"laymen."  Thus,  the  schism  between  the 
"shrink"  and  the  "bean  counter."  The  goal  of 
the  present  paper  is  to  develop  a quantitative 
and,  more  importantly,  a communicable  frame- 
work for  establishing  training  effectiveness 
criteria  that  can  be  used  and  understood  by 
both  the  training  specialist  and  the  manager. 

Training  Decisions 

In  the  process  of  developing  a training 
program,  an  obvious  question  relates  to  which 
behaviors  (tasks)  will  be  trained  and  which 
will  not.  That  is,  for  any  nontrivial 
operator's  job,  it  is  impossible  to  exhaust 
the  possible  contingencies  that  might  arise. 

In  fact,  humans  are  often  included  precisely 
to  utilize  their  conceptual  generalization 
capabilities.  The  only  point  here  is  that 
one  decision  that  must  be  made  is  whether  or 
not  to  train  on  a particular  task.  To  date, 
there  has  been  no  algorithm  established  to 
assist  in  this  decision. 

A second  decision  which  is  an  extension  of  the 
same  issue  is,  if  a task  is  to  be  trained, 
what  level  of  proficiency  should  be  required? 

That  is,  what  should  be  the  training  duration 
(which  the  manager  translates  into  dollars). 

It  is  well  known  that  the  rate  of  learning 
exhibits  diminishing  returns  as  a function  of 
training  time. 

A third  question,  that  has  become  prominent 
in  military  training,  is  related  to  the  com- 
plexity (cost)  of  training  devices.  As  with 
the  relation  between  learning  rate  and  time, 
there  are  diminishing  returns  of  learninq  rate 
as  the  complexity  of  the  training  device  surpass- 
es a particular  level.  Figure  1,  illustrates 


the  relationship  of  learning  rate  and  device 
complexity  (cost).  The  “bean  counter"  is 
obviously  interested  in  establishing  where 
the  additional  cost  of  training  (time  and/or 
devices)  is  not  warranted.  This  paper  will 
present  an  approach  to  establishing  the 
required  information  necessary  to  make  the 
above  decisions  and  a quantifiable  format  of 
this  information. 


Figure  1.  Cost  and  Training  Effectiveness 
as  a Function  of  Training  Device 
Complexity 


The  Quality  Control  Aspect  of  Training 
Criteri  a 

Training  programs  can  be  viewed  as  means  of 
increasing  the  reliability  (i.e.  quality)  of 
one  of  the  components  in  the  man-machine 
system.  Therefore,  the  "training  criteria" 
(time,  performance  levels,  etc.)  are  analo- 
gous to  the  quality  levels  utilized  in 
industrial  quality  control  schemes.  There- 
fore, the  factors  that  are  involved  in 
establishing  quality  levels  in  a Bayesion 
economic  analysis  model  (costs,  payoffs,  and 
probabilities)  are  appropriate  to  establish- 
ing the  cost-effectiveness  of  training 
criteria  on  the  basis  of  a Loss  Function. 

The  goal  of  this  aoprp»ch  is  to  determine  the 
out-going  level  of  the  trainee  on  the  basis 
of  minimizing  total  costs  that  relate  to  that 
quality  level . 
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METHODOLOGY 

The  general  approach  discussed  in  this  paper 
involves  viewing  training  as  a means  of 
increasing  the  reliability  (quality)  of  the 
human  operator.  The  training  criterion 
(outgoing  quality)  is  established  within  the 
context  of  decision  theory  parameters  (costs, 
payoff,  and  probabilities). 

Increasing  Reliability  Through  Training 

Although  it  is  seldom  discussed  in  this  con- 
text, training  is  simply  a means  of  increas- 
ing the  reliability  of  one  of  the  components 
in  a man-machine  system.  In  the  same  sense 
that  increasing  the  quality  of  any  other 
component  (in  terms  of  material  or  workman- 
ship) increases  reliability,  the  quality  of 
the  human  component  is  increased  through 
training.  There  are  both  advantages  provided 
by  the  increased  reliability  and  costs  incur- 
red by  the  increase. 

Training  criteria  are  the  means  by  which  the 
quality  of  the  human  product  is  assured.  In 
the  same  sense  as  the  producers'  and  consumers' 
risk  are  variables  in  a quality  control 
scheme,  the  amount  of  risk  (cost)  involved 
with  undertraining  or  overtraining  a human 
is  also  a variable. 

The  "natural"  quality  of  the  process  is  an 
important  aspect  to  consider  in  quality 
control  schemes.  Similarly,  the  "natural" 
reliability  of  the  human  for  various  tasks  is 
important  in  establishing  the  training  cri- 
teria. There  is  an  extensive  amount  of 
research  literature  on  the  topic  of  human 
reliability.  This  information  is  often 
"descriptive"  of  the  human  (e.g.,Askren  and 
Regulinski,  1969;  and  Beck,  Hayman,  and 
Markisohn,  1967);  but  it  also  includes  infor- 
mation that  is  "prescriptive"  in  nature 
(Feherman  and  Siegel,  1973;  Siegel,  Wolf, 
and  lautman,  1974;  and  Sontz  and  Lamb,  1975). 
These  studies  have  resulted  in  an  extensive 
methodology  for  assessing  human  reliability; 
however,  there  is  little  direct  information 
as  to  increases  in  human  reliability  through 
training  (Goldman  and  Slattery,  1964).  This 
relationship  must  be  inferred  from  the  reli- 
ability data  in  combination  with  information 
about  learning. 

Learning  Rate 

The  literature  on  reliability  and  human 
performance  provides  extensive  data  and  a 
methodology  for  estimating  human  performance 
parameters.  That  is,  the  range  of  the  poten- 
tial "outgoing  qualities"  for  various  types 
of  tasks  can  be,  and  in  some  cases  has  been, 
established  (within  rather  large  bounds). 

The  question  remains,  however,  as  to  the  rate 
of  quality  increase  (learning)  as  a function 
of  time.  In  fact,  there  Is  a significant 


amount  of  information  pertaining  to  learning 
rates  for  both  discrete-serial  and  continu- 
ous perceptual -motor  tasks  (e.g., Fleishman, 
and  Parker,  1962;  and  Neumann  and  Ammors , 
1957).  For  various  types  of  tasks,  the 
learning  rate  versus  time  function  (the 
learning  curve)  can  be  specified  with  rela- 
tive confidence.  In  fact,  the  stability  of 
the  learning  curve  has  resulted  in  severe 
difficulties  in  demonstrating  effects  due  to 
different  "training  techniques."  That  is, 
for  a given  task,  trainees  learn  "in  spite 
of  the  training  technique."  For  the  purposes 
at  hand,  this  is  a significant  advantage. 
Figure  2 illustrates  the  general  shape  of  the 
learning  curve. 


Figure  2.  Typical  Learning  Curve 


Information  Requirements 

As  discussed  in  the  previous  sections,  there 
is,  to  a greater  or  lesser  degree,  an  exten- 
sive literature  base  pertaining  to  human 
reliability  and  learning  rate  as  a function 
of  time.  The  present  section  addresses  the 
other  information  that  is  required  to  estab- 
lish an  outgoing  quality  level  of  a training 
program  on  the  basis  of  economics. 

The  first  (and  in  the  extreme,  the  most 
obvious)  parameter  is  the  probability  that 
the  operator  will  ever  perform  the  particular 
task.  For  example,  emergency  procedures  to 
handle  abnormal  situations  often  occur  with 
a relatively  low  probability.  Intuitively 
(and  pragmatically)  there  is  a low  probabil- 
ity of  occurrence  beyond  which  training  on 
that  task  would  not  be  "justified."  Simi- 
larly, on-  the  other  end  of  the  continuum,  for 
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a task  that  occurs  routinely,  training  could 
probably  be  "justified."  The  point  is  that 
(as  with  the  relationship  between  the  probab- 
ility of  a defect  and  the  requirements  for  a 
quality  control  scheme)  the  existence  or 
extent  of  training  on  a task  is  affected  by 
the  probability  of  that  task  occurring. 

A second  set  of  parameters  that  is  required 
for  an  economically  based  decision  is  that  of 
costs.  There  are  two  general  categories  of 
cost.  One  category  involves  the  costs  of 
training.  These  costs  include  oersonnel  costs 
(trainees,  instructors,  managers),  training 
equipment  procurement  costs  (from  textbooks 
to  complex  system  simulators),  and  operation 
and  maintenance  costs  (including  those  asso- 
ciated with  facilities).  Another  category 
of  costs  that  is  important  in  establishing 
training  criteria  involves  the  cost  of  a 
failure  to  perform  the  operational  task. 
Included  in  this  category  are  such  things  as 
costs  due  to  waste  when  an  operator  allows  a 
process  to  go  out  of  control,  equipment 
damage,  and  loss  of  life  or  personal  injury. 

Obviously,  some  of  these  costs  are  more  easily 
arrived  at  than  others.  For  example,  the 
"cost  of  human  life  or  injury"  is  potentially 
very  difficult  to  ascertain.  However,  in 
terms  of  law  suit  settlements,  these  numbers 
are  being  established.  In  addition,  tacti- 
cians have  been  establishing  equally  difficult 
costs  in  war-game  simulations  that  are  used 
to  evaluate  a weapon  system’s  effectiveness. 
The  point  to  be  made  here  is  that,  although 
assessing  (or  at  least  asserting)  costs  is 
often  difficult,  it  is  possible.  A second 
point  is  that  "establishing"  these  costs  is 
not  within  the  technical  area  of  the  training 
specialist  and  cannot,  therefore,  be  his  res- 
ponsibility. This  is  analogous  to  the  costs 
in  the  economic  decision  process  of  estab- 
lishing outgoing  quality  levels  in  any 
quality  assurance  situation.  It  cannot  be 
the  quality  control  specialist's  responsi- 
bility to  determine  the  costs  of  "unaccept- 
able" quality. 

The  last  set  of  parameters  that  impacts  upon 
training  criteria  decisions  involves  the  pay- 
offs afforded  through  training  as  a function 
of  time.  That  is,  what  are  the  probabilities 
that  the  human  will  "handle"  the  situation 
given  that  the  situation  does  occur.  Simi- 
larly, how  does  this  probability  vary  as  a 
function  of  training  time.  These  questions 
relate  to  the  previously  discussed  human 
reliability  estimates  and  the  learning  curve. 
Whereas,  the  probability  of  a situation 
occurring  and  the  costs  involved  are  the 
responsibility  of  managers  (with  the  counsel 
of  experts  in  reliability  and  economics),  this 
aspect  of  the  information  necessary  for  a 
decision  is^  within  the  technical  expertise  of 
the  training  (human  performance)  specialist. 
Now  that  the  categories  of  information  have 


been  established,  the  following  section  of 
this  paper  will  address  the  computation 
involved  in  the  decision  "process." 

COMPUTATIONS 

The  purpose  of  this  section  is  to  present  a 
"first-cut"  algorithm  that  is  potentially 
useful  in  making  training  criteria  decisions. 
First,  it  is  necessary  to  establish  the 
notation  that  will  be  used  and  define  the 
meaning  of  the  parameters. 

P - probability  that  the  situation  will 
occur. 

P - probability  of  performing  the  task 
without  training,  given  that  the 
situation  occurred. 

Pt  - probability  of  performing  the  task 
after  training  time  t,  given  that 
the  situation  occurred. 

- cost  of  a unit  of  training  time. 

K - cost  of  a failure  to  perform  the 
occ  task. 

The  increase  in  the  probability  of  the  human 
to  perform  a task  after  training  time  t is: 

(P.  - P ) (P  ) 

' t o'  occ' 

The  pay-off  value  contributed  by  the 
training  is: 

(pt  - Po>  <Pocc>  <Kocc> 

The  cost  of  training  for  a period  of  time  t 
is: 

(KM)  (t) 

Therefore,  "the  break-even  point"  in  terms  of 
the  training  criterion  is  when  the  pay-off 
value  of  the  training  is  equal  to  the  cost  of 
the  training.  That  is,  when: 

<Pt  - Po>  <Pocc>  <Kocc>  , 

1 • 

When  the  left  side  of  this  equation  is 
greater  than  1,  additional  training  is  cost- 
effective;  when  it  is  less  than  1,  it  is  not. 

Manipulating  this  equation  results  in: 

(pt  - V (KAt) 
f ‘ (Pocc>  («occ> 

As  discussed  in  the  previous  section,  pro- 
viding the  data  for  the  right-hand  side  of 
the  equation  is  the  responsibility  of  the 
combination  of  manager,  reliability  expert, 
and  the  economist.  The  information  pertain- 
ing to  the  right-hand  side  is  the  responsi- 
bility of  the  training  specialist. 
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Recall,  from  the  previous  section  that  the 
increment  in  rate  of  learning  is  not  a 
linear  function  of  training  time.  Therefore, 
the  optimization  process  of  the  decision 
involves  increasing  training  time  (t)  to  the 
point  where  the  equation  is  satisfied.  Fig- 
ure 3,  graphically  illustrates  this  procedure. 


Figure  3.  Function  Relating  Training 
Decision  Parameters 


HYPOTHETICAL  EXAMPLES 

For  the  purpose  of  illustrating  the  method, 
a simple  example  will  be  presented.  Assume 
that  a training  program  is  being  initiated 
in  a widget  factory.  There  is  a particular 
situation  that  can  lead  to  a malfunction  that 
approximately  one  out  of  twenty  operators 
experience  (e .g. .average  tenure  in  the  job 
of  5 years,  probability  of  situation  occur- 
ring is  0.01  during  any  particular  year).  If 
the  situation  occurs,  the  operator  must  per- 
form a series  of  steps  in  a procedure  within 
a short  period  of  time.  If  he  fails,  the 
widget-making  machine  explodes  with  a replace- 
ment cost  of  $16,000.  The  total  cost  of  one 
unit  of  training  time  is  $35.00.  In  addition, 
assume  that  the  learning  rate  is  approximated 
by  the  curve  found  in  the  study  by  tleishman 
and  Parker  (1962).  The  question  is:  "How 

long  should  the  operator  be  in  training?" 

The  parameter  values  for  this  example  are: 

K^t  * 35 ; Kocc  * 16’°°0;  and  Pocc  " -05- 

Figure  4 illustrates  the  function,  P.  - P s 
f (t)  and  Table  1 gives  discrete  values 
of  P1  - p , t,  and  (Pt  - p )/t . 


Figure  4.  Learning  Curve  Adapted  from 
Fleishman  and  Parker  (1962) 


Table  1 
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Therefore,  it  can  be  determined  from  the 
table  that  approximately  15  time  units  of 
training  is  optimal . 

A second  example,  in  a different  context,  is 
an  overcoat  manufacturing  factory  in  which 
there  is  a training  program  for  seamstresses. 
In  this  case,  the  Pgrr  is  the  probability 
that  the  customer  will  decide  upon  buying 
the  coat  on  the  basis  of  the  sewing  (as 
opposed  to  the  materia'  or  design).  The  K 
is  the  lost  revenue  if  the  customer  does  c 
not  by  the  coat.  In  this  case,  however,  the 
P^  is  the  probability  that  the  customer  will 
accept  the  coat,  given  that  the  sewing  is 
one  of  his  decision  points.  Through  training 
the  Pt  value  increases  as  in  any  other 
learning  situation.  Given  the  required  cost 
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data,  probabilities,  and  learning  curve,  the 
extent  of  training  that  is  cost-effective  can 
be  determined. 

The  purpose  of  the  present  paper,  and  the 
examples  given,  is  to  demonstrate  the 
feasibility  of  approaching  training  require- 
ments in  terms  of  decision  theory  parameters. 
The  next  section  discusses  the  next  steps  in 
refining  this  approach. 

NEXT  STEPS 

An  integral  part  of  the  approach  described 
in  this  paper  is  the  use  of  the  learning 
curve  characteristics  in  determining  training 
criteria.  It  was  stated  earlier  that  we  have 
an  extensive  amount  of  information  pertaining 
to  the  shape  of  the  learning  curve  for  many 
tasks.  However,  what  is  needed  for  this 
approach  (as  well  as  many  other  aspects  of 
human  performance)  is  a descriptive  "taxonomy 
of  tasks"  that  vary  in  their  learning  charac- 
teristics. This  taxonomy  could  be  used  by 
training  learning  specialists  to  estimate  the 
function  P = f ( t ) for  the  tasks  that  make  up 
their  particular  operational  job.  There  have 
been  selected  efforts  in  the  area  of  task 
taxonomy  (e.g.  Miller,  1971).  In  fact, 
Meister  and  Mills  (1971)  and  Mills,  Bachert, 
and  Hatfield  (1975)  have  categorized  tasks  in 
terms  of  human  reliability.  Swain  (1963, 
1974)  has  also  demonstrated  this  possibility. 
Although  this  work  does  not  provide  learning 
curve  information,  it  does  serve  to  set  the 
boundaries  in  terms  of  human  performance 
reliability. 

The  actual  calculations  required  by  this 
methodology  could  be  significantly  simplified 
by  applying  an  appropriate  transformation  to 
the  curve  that  would  result  in  a linear  rela- 
tionship. For  example,  a logarithmic  trans- 
formation is  the  obvious  starting  point  for 
this  type  of  effort. 

In  addition  to  increasing  the  confidence  in 
the  learning  data  and  simplifying  the  compu- 
tation requirements,  a major  "next  step"  is 
to  determine  the  sensitivity  of  the  various 
parameters.  This  type  of  analysis  will  allow 
one  to  allocate  his  time  appropriately  in 
terms  of  refining  the  information  to  which 
the  cost-effectiveness  is  most  sensitive. 

IMPLICATIONS  OF  THE  METHODOLOGY 

One  of  the  more  interesting  implications  of 
the  proposed  methodology  is  that  it  illus- 
trates the  absurdity  of  the  "perfect  mastery" 
approach  on  a cost-effectiveness  basis.  That 
is,  on  the  basis  of  costs  and  payoffs,  it  is 
the  rate  of  learning  as  a function  of  time 
that  is  the  important  metric.  The  appro- 
priate criterion  for  terminating  training 
for  a particular  individual  is  when  his 


increased  proficiency  gain  per  unit  time 
(i .e. , dollars)  is  less  than  the  payoff 
contributed  to  operational  success.  This 
approach  is  compatible  with  the  techniques  of 
Computer  Managed  Instruction  (or  adaptive 
training  in  perceptual -motor  skill  training) 
in  that  the  rate  of  proficiency  increase  is 
accessible.  The  point  is  that  an  individual's 
present  proficiency  is  not  the  only  criterion 
variable;  it  is  also  important  to  evaluate 
his  rate  of  advancement  and  subsequently 
estimate  his  potential  for  further  advance- 
ment. 

Another  important  implication  of  this  method- 
ology is  the  categorization  of  the  types  of 
information  necessary  to  make  valid  training 
system  design  decisions  and  the  assignment 
of  responsibilities  for  the  various  informa- 
tion. Decisions  relative  to  training  cri- 
teria are  the  joint  responsibi  1 i ty  of 
operational  personnel,  managers,  and  training 
specialists. 

The  problems  of  establishing  training 
criteria  are  very  complex;  however,  there 
must  be  a framework  from  which  to  start  if 
progress  is  to  be  made.  The  present  paper 
discusses  the  applicability  of  techniques 
presently  developed  in  the  areas  of  quality 
assurance,  training  technology,  and  decision 
theory  to  the  problem  of  establishing  cost- 
effective  training  criteria.  The  types  of 
data  required,  and  the  appropriate  respon- 
sibilities for  those  data  are  discussed. 

There  is  no  apparent  reason  why  the  cost- 
effectiveness  of  training  cannot  be  estab- 
lished through  the  same  methods  as  have  been 
used  for  other  systems  for  many  years  and 
with  significant  methodological  advances. 
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psychology  included  visual  and  auditory  perception,  information  processing,  and 
sensory  physiology.  He  received  the  M.S.  degree  at  University  of  Illinois  Aviation 
Research  Laboratory  and  is  doing  research  for  the  Ph.V.  dissertation  in  Human 
Factors  at  State  University  of  New  York  at  Buffalo. 
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DO-963  CLASS  DESTROYER 

ENGINEERING  CONTROL  AND  SURVEILLANCE  SYSTEM  TRAINER 

H.  C.  ROBINSON,  JR. 

THE  SINGER  COMPANY,  LINK  DIVISION 
SILVER  SPRING  OPERATION 


The  Navy  requires  trained  crews  for  maintain  student  safety.  For  example,  con- 

operation  and  maintenance  of  the  DD-963  sole  requirements  for  high  voltage  current 

Destroyer  Engineering  Control  and  Surveil-  were  converted  to  accept  standard  analog  sig- 

lance  System  (ECSS).  Engineering  crew  train-  nals  for  obvious  safety  reasons.  This  type 

ing  onboard  ship  is  not  optimum  in  terms  of  of  hardware  is  suitable  for  either  simulation 

personnel  safety,  equipment  readiness  re-  or  stimulation  approach, 

quirements,  fuel  conservation,  environmental 

considerations,  manpower  assignment,  and  The  consoles  used  on  the  ECSS  trainer 

other  logistical  considerations.  The  new  are  provided  by  the  Navy  from  their  opera- 

and  highly  automated  design  of  the  DD-963  tions  equipment  stores.  It  is  through  these 

Class  destroyer  and  the  limited  availability  consoles,  configured  exactly  as  their  actual 

of  these  ships  for  training  engineering  crews  shipboard  counterparts,  that  ships'  systems 

emphasize  the  use  of  an  alternative  training  are  "stimulated"  to  make  the  traininq  envi- 

approach.  The  DD-963  Class  destroyer  ECSS  ronment  identical  to  conditions  on  the  real 

trainer  provides  effective  operation  and  ship, 

maintenance  training  in  a wide  spectrum  of 

situations  that  cannot  be  practically  per-  Operational  Traininq 

formed  under  normal  operations  on  board  ship. 

Capabilities  of  software  simulation  models  The  trainee  develops  operating  skills 

create  a trainer  that  can  accurately  duplicate  through  hands-on  experience  in  both  routine 
ship  conditions  from  cold  steel  to  cruising.  and  emergency  engineering  procedures.  The 

With  a versatile  training  device  of  this  type,  student  is  familiarized  with  the  positioning 

ECSS  crews  --  through  practical  experience  in  and  function  of  the  controls  and  indicators 

operations  and  maintenance  --  can  be  thor-  of  each  trainer  console  --  the  similarity 

oughly  prepared  for  virtually  every  aspect  of  between  trainer  and  ship  being  emphasized 

their  particular  assignments  onboard  ship.  (see  figure  1).  This  familiarization  process 

could  not  feasibly  be  accomplished  on  an 

The  systems  simulated  in  the  trainer  actual  ship  because  of  the  level  of  complex- 

accurately  recreate  ship  performance  and  ity  of  operating  modes  and  because  of  the 

generate  signals  to  "stimulate"  actual  ship-  sheer  quantity  of  components  on  each  panel, 

board  equipment,  namely:  For  example,  the  Propulsion  and  Auxiliary 

Control  Console  (part  of  PAMCE)  alone  in- 

• Electric  Plant  Control  Equipment  eludes  some  1,000  discrete  indicators 

(EPCE)  (alarms/status)  and  400  switches. 

• Propulsion  and  Auxiliary  Machinery  The  principal  area  of  procedures  training 

Control  Equipment  (PAMCE)  centers  on  the  control  and  monitoring  of  the 

ship's  propulsion  gas  turbines.  Control  and 

• Propulsion  and  Auxiliary  Machinery  monitoring  is  handled  from  the  Propulsion  and 

Information  System  Equipment  (PAMISE)  Auxiliary  Control  Console  and  from  the  Pro- 
pulsion Local  Control  Console  (part  of  PL0E). 

• Propulsion  Local  Operating  Start/stop  procedures  for  the  gas  turbines 

Equipment  (PL0E)  are  drilled  in  all  control  modes,  manual, 

manual -initiate,  and  auto-initiate.  In  addi- 

The  heart  of  the  simulator/trainer  is  a tion  to  normal  start/stop  procedures,  emer- 

medium-capacity  high-speed  minicomputer  with  gency  stops,  false  starts,  internal  fires, 

28K  of  16-bit  memory  words.  Software  models  and  module  fires  are  simulated.  This  famil- 

residing  and  executing  in-core  memory  com-  iarizes  the  student  with  such  occurrences  and 

municate  with  the  actual  shipboard  consoles  drills  proper  remedial  action.  The  simulated 

through  two  distinct  types  of  interface  propulsion  turbines  can  also  be  operated  at 

equipment.  Analog  signals  (-10  vdc  to  +10  all  cruise  speeds.  To  reach  a cruise  config- 

vdc)  and  discrete  signals  (0  vdc  and  28  vdc)  uration,  the  propulsion  turbines  must  be 

are  received  and  transmitted  through  a coupled  to  the  propeller  shaft  through  the 

standard  hybrid  interface.  Serial  infor-  main  reduction  gear.  The  trainee  is  dili- 

mation  is  fed  to  and  tapped  from  the  gently  exercised  in  the  reduction-gear  clutch 

interconsole  serial  data  bus.  Console  and  power-turbine  brake  operations  necessary 

alteration  has  been  kept  at  an  absolute  to  safely  accomplish  this  coupling.  This 

minimum  to  ensure  training  accuracy  and  to  could  simply  involve  one  action  if  the  auto- 
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ELECTRIC  PLANT 
CONTROL  CONSOLE 
(EPCC) 


ECSS  Training  Device  (DD-963  Class) 


initiate  mode  is  selected,  or  it  could  involve 
several  brake  releases  and  clutch  engagements 
to  achieve  a full-power  or  split-plant 
configuration. 

Other  training  areas  include  monitoring/ 
controlling  of  the  gas  turbine  generators 
from  the  Electric  Plant  Control  Console  (part 
of  the  EPCE).  In  addition  to  normal  start/ 
stop  procedures,  trainees  learn  electric  bus- 
power  configuring,  automatic-  and  manual-load 
shedding,  generator  paralleling,  and  bus- 
loading. The  interaction  between  shore  power 
and  ship  power  is  another  basic  simulation. 
This  familiarizes  the  trainee  with  the  capa- 
bilities of  shore  power  operation  and  teaches 
procedures  for  paralleling  generators  to 
shore  power.  Special  features  have  been  in- 
corporated into  the  simulation  program  as  a 
result  of  data  received  from  builder's  trials. 
One  such  addition  is  the  capability  to  break 
a generator  shaft  from  the  control  console  if 
the  proper  conditions  exist.  This  is  surely 
an  important  training  item  when  one  considers 
the  danger  to  numan  life  and  the  high  cost  of 
material  replacement  of  such  an  action  on 
board  the  real  ship. 

Interconsole  propulsion  information  is 
transmitted  via  the  Engine  Order  Telegraph 
(EOT)  system.  In  automatic-throttle  mode, 
the  EOT  system  transmits  standard  orders 
(Flank  3,  Flank  2,  Flank  1,  Full,  Standard, 
Ahead  2/3,  Ahead  1/3,  Stop,  etc.)  to  the 
appropriate  station  in  control.  In  manual- 
throttle  mode,  the  EOT  system  transmits  com- 
manded shaft  RPM  and  propeller  pitch.  The 
instructor  in  the  simulated  bridge  control 
station  communicates  plant  mode  requirements 
(full  power,  split  plant,  secure),  in  addition 
to  station-in-control  commands  (pilot  house, 
central  control  station),  to  the  Propulsion 
Auxiliary  Control  Console  via  the  EOT  system. 
All  of  these  operations  require  exacting  care 
in  initiating  and  acknowledging  procedures 
for  errorless  communication. 

Vital  auxiliary  systems  --  such  as  fuel 
oil,  lube  oil,  controllable-reversible  pitch 
(CRP)  propeller,  and  bleed  air  systems  --  are 
simulated.  Procedures  for  fuel  and  lube  pump 
operation  and  alignment  are  drilled  in  all 
operating  modes  (A-lead,  B-lead,  and  manual). 
Valve  alignment  requirements  are  simulated  in 
the  bleed  air  system  to  instruct  proper  valve 
positioning  for  various  conditions  --  low  and 
high  pressure  starter  air  (propulsion  turbines 
and  generator  turbines),  masker  air  (used  to 
create  a laminar  bubble  screen  beneath  each 
engine  room  to  attenuate  ship  radiated  noise), 
prairie  air  (used  to  reduce  propeller  noise), 
and  air  for  anti-icing  (used  to  warm  intake 
air  to  prevent  condensed  water  on  the  de- 
misters from  freezing).  Since  the  shafts  in 
the  DO-963  Class  destroyer  rotate  in  a uni- 
directional manner,  the  direction  of  ship 
motion  is  dependent  on  propeller  pitch. 


Because  of  the  obvious  importance  of  the  CRP 
propeller  system,  students  are  thoroughly 
exercised  in  propeller  pitch  control.  Non- 
vital  auxiliary  systems,  such  as  the  ship's 
service  air  system,  high  pressure  air  system, 
waste  heat  boilers,  distillation,  and  sewage 
system,  are  also  simulated.  Even  though 
these  systems  do  not  have  extensive  controls 
from  the  Propulsion  Local  Control  Console  or 
the  Propulsion  and  Auxiliary  Control  Console, 
normal  operations  can  be  verified  and  correc- 
tive procedures  can  be  reviewed  for  emergency 
situations. 

Operation  of  the  Propulsion  and  Auxiliary 
Machinery  Information  System  Equipment  (PAMISE) 
is  another  important  training  area.  The  PAMISE 
is  responsible  for  gathering  information  ship- 
wide from  the  PAMCE,  PLOE,  and  EPCE,  and  then 
processing  this  data  in  the  Central  Information 
System  Equipment  (CISE)  computer.  This  infor- 
mation is  made  available  to  console  operators 
via  the  demand  display  indicators  located 
throughout  the  consoles.  An  operator  can  moni- 
tor hundreds  of  ship  parameters  by  simply  dial- 
ing an  appropriate  address.  The  student  learns 
to  program  the  CISE  computer,  and  he  becomes 
proficient  in  computer  peripheral  operation 
(bell  logger,  alarm  logger,  and  tape  reader). 

Since  no  piece  of  equipment  can  be  ex- 
pected to  function  perfectly  at  all  times, 
the  ECSS  trainer  also  provides  for  practice 
in  problem  identification  and  resolution. 

The  instructor  can  input  malfunctions 
affecting  almost  every  aspect  of  ship  per- 
formance. These  malfunctions  simulate  nor- 
mal operating  problems  that  drill  the  trainee 
in  recognizing  trouble  and  reacting  to  emer- 
gency operations.  Some  malfunctions  are 
terminal  --  that  is,  not  correctable  by  the 
console  operator;  others  can  be  circumvented 
if  the  trainee  takes  proper  actions. 

One  typical  "trainee-correccab'ie"  mal- 
function is  loss  of  low  pressure  air  for 
starting.  The  result  is  that  high  pressure 
start  air  is  required  or  the  instructor  can 
initiate  malfunction  of  the  fuel  oil  heater 
to  prevent  warm  fuel  from  being  delivered 
to  the  propulsion  turbines.  The  trainee, 
realizing  the  situation,  learns  to  activate 
the  cold-fuel  override  switch  for  emergency 
starting.  Lube  or'fuel  pump  failures  prompt 
the  student  to  staft  backup  activation.  CRP 
propeller  pump  fa/lure  requires  shaft  pump 
backup.  Loss  of  generator  turbine  power 
requires  an  electric  load  change  until  the 
system  can  be  properly  reconfigured.  Gas 
turbine  generator  low  frequency  or  low  vol- 
tage malfunctions  require  appropriate  adjust- 
ments to  correct  the  condition. 

A typical  terminal  malfunction  is  the 
lube  oil  fuel  oil  strainer  differential  pres- 
sure high  alarm.  The  student  (as  console 
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operator)  learns  to  remedy  the  situation  by 
contacting  the  instructor  (as  local  operator) 
ordering  a filter  change.  With  the  insertion 
of  a journal -bearing  temperature  high  mal- 
function or  a thrust-bearing  temperature  high 
malfunction,  the  student  realizes  that  lube 
oil  is  not  being  supplied  in  proper  quantities. 
This  requires  bringing  the  shaft  to  a stop 
to  reduce  further  damage  and  ordering  repairs. 
These  types  of  simulated  emergency  conditions 
cannot  practically  or  safely  be  created  on- 
board ship. 

Maintenance  Trai ning 

In  addition  to  training  in  operating 
procedures,  the  ECSS  trainer  provides  the 
student  with  hands-on  experience  in  mainte- 
nance of  actual  shipboard  equipment.  Train- 
ees become  intimately  familiar  with  all  com- 
ponents of  the  EPCE , PAMCE,  PAMISE,  and  PLOE. 

To  develop  combat-ready  skills,  students 
are  drilled  in  rapid  fault  isolation.  This 
may  be  as  simple  as  locating  and  replacing  a 
malfunctioning  printed  circuit  card,  or  it 
could  involve  tracing  signals  from  point  of 
console  entry  to  point  of  use  or  display. 
Printed  circuit-card  calibration  is  presented 
to  sharpen  skills  in  standard  routine  main- 
tenance. This  requires  removing  particular 
cards  from  the  console,  positioning  them  in 
proper  tests  slots,  and  performing  the  appro- 
priate test.  The  majority  of  information 
transmitted  between  consoles  is  in  serial 


format.  Monitoring  this  information  with  an 
oscilloscope,  locating  faults,  verifying  con- 
tent, and  checking  voting  logic,  are  all 
tasks  which  the  student  learns  for  this 
serial  data  bus. 

Other  maintenance  areas  involve  non- 
electrical components.  The  student  learns  to 
assemble  and  disassemble  such  items  as  the 
integrated  throttles,  power  level  angle  (PLA) 
and  pitch  quadrants,  power  supplies,  and  the 
like.  The  instructor  can  simulate  hardware 
malfunctions  in  the  equipment  --  for  example, 
inserting  a bad  connector,  a bad  card,  or 
shorting  two  pins  together.  Since  the  train- 
ing device  gets  more  intense  usage  than  does 
the  actual  shipboard  equipment,  repair  of  un- 
planned problems  constitute  another  important 
aspect  of  training.  This  type  of  'trouble- 
shooting" learning  is  excellent  preparation 
for  ship  duty. 

As  the  first  simulator/trainer  of 
a gas  powered  turbine  ship,  the  DD-963 
Class  Destroyer  ECSS  trainer  provides 
an  ideal  training  environment  for 
maintenance  of  the  actual  shipboard 
equipment.  The  first  delivered  unit 
is  currently  located  and  operational 
at  the  Great  Lakes  Naval  Training 
Center  where  it  is  utilized  approxi- 
mately 22  hours  per  day.  This 
utilization  has  produced  200  operator 
personnel  and  50  maintenance  personnel 
during  the  first  year  of  operation. 
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THE  VOICE  DATA  COLLECTION  PROGRAM 
A GENERALIZED  RESEARCH  TOOL  TOR  STUDIES  IN  SPEECH  RECOGNITION 

ROBERT  BREAUX 

Naval  Training  Equipment  Center 

MICHAEL  W.  GRADV 
Logicon,  Inc. 


INTRODUCTION 

The  technology  of  machine  speech  under- 
standing, integrated  with  advanced  instruc- 
tional technology,  has  demonstrated  the  capa- 
bility to  automate  training  for  tasks  which 
are  characterized  by  the  use  of  restricted, 
stylized  speech.  Such  a training  system  for 
the  GCA  Controller  has  been  described  at  NTEC/ 
Industry  Conference s in  1974  and  1975.  >2 
This  paper  discusses  the  development  since  that 
time  of  a more  generalized  laboratory  research 
system  designed  to  support  speech  recognition 
studies  for  evaluation  of  essentially  any  vo- 
cabulary set.  Moreover,  the  program  provides 
the  experimenter  with  sufficient  flexibility 
to  investigate  training  the  student  on  the 
vocabulary  itself;  that  is,  the  critical  task 
of  teaching  proper  phraseology  to  the  student, 

BACKGROUND 

The  advantages  of  automated  adaptive  in- 
struction are  well  known  to  the  training  com- 
munity. Tasks  requiring  verbal  commands  and 
instructions  had,  until  recently,  been  unamen- 
able to  these  automated  training  techniques. 
Performance  in  speech-based  tasks  was  tradi- 
tionally measured  by  subjective  instructor 
ratings.  With  the  advent  of  the  speech  recog- 
nition capability,  however,  an  individualized, 
self-pacing,  automated  training  system  for 
these  tasks  (now  including  objective  perform- 
ance measurement)  became  feasible. 

The  Naval  Air  Systems  Command  has  sup- 
ported the  Naval  Training  Equipment  Center's 
( NAVTRAEQU I PCEN ) Human  Factors  Laboratory 
in  efforts  to  establish  design  guidelines  for 
such  speech  understanding-based  systems.  In- 
deed, the  basic  feasibility  of  these  systems 
was  demonstrated  in  a NAVTRAEQUIPCEN  laboratory 
version  of  a Ground  Controlled  Approach  Con- 
troller Training  System  (GCA-CTS)  developed 
under  contract  with  Logicon,  Inc.’>' 

LABORATORY  R&D,  AND  THE  VOICE 
DATA  COLLECTION  PROGRAM 

Tasks  such  as  those  of  the  Landing  Signal 
Officer,  Air  Intercept  Controller,  and  Ship 
Conning  Officer  would  also  appear  amenable  to 
a speech  understanding-based  automated  training 
system.  Future  R&D  should  proceed  to  address 
these  new  areas.  The  GCA  Controller  has  served 


as  a test  application;  now  other  vocabularies 
(applications)  must  be  studied. 

Furthermore,  the  problem  of  training  the 
student  in  correct  pronunciation  and  use  of  the 
radio  terminology,  operational  brevity  code, 
"standard  commands,"  etc.  is  itself  a subject 
for  study.  At  least  a basic  knowledge  of  the 
relevant  vocabulary  is  required  prior  to  train- 
ing for  the  task  which  employs  the  phraseology. 
R&D  efforts  can  be  wisely  directed  along  these 
1 ines  as  well . 

Finally,  experience  with  the  laboratory 
GCA-CTS  has  highlighted  certain  risk  areas 
associated  with  the  recognition  of  some 
phrases.  Further  R&D  analysis  of  these  areas 
with  other  vocabularies  can  support  the  defi- 
nition of  future  training  programs,  identify 
problem  areas  early  in  the  development  cycle, 
and  in  general  support  the  establishment  of 
design  guidelines. 

Aware  of  each  of  these  requirements,  NAV- 
TRAEQUIPCEN has  supported  the  development  of  a 
highly  flexible  laboratory  research  too 1 called 
the  Voice  Data  Collection  (VDC)  program.  The 
VDC  program  provides  the  framework  around 
which  the  researcher  can: 

a.  Define  vocabulary  phrases  associated 
with  essentially  any  application. 

b.  Preprogram  the  presentation  of 
phrases  or  groups  of  phrases  to  the  speaker 
via  text,  graphics,  and/or  computer  synthe- 
sized speech,  resulting  in  an  effective  envi- 
ronment in  which  to  learn  the  vocabulary 
phrases. 

c.  Test  the  ability  of  existing  hard- 
ware/software algorithms  to  recognize  these 
phrases,  and  extract  hardcopy  data  on  recog- 
nition reliability,  correlation  scores,  and 
potential  system  confusions. 

As  such,  the  VDC  program  specifically  ad- 
dresses the  three  areas  previously  identified 
as  possible  directions  of  future  R&D.  The 
remaining  sections  of  this  paper  are  a brief 
description  of  these  aspects  of  the  VDC  pro- 
gram. A more  complete  explanation  of  VDC 
functions  and  user  capabilities  is  available 
in  the  references. 4 


229 


■ 


VOCABULARY  DEFINITION 


operation.  It  was  difficult  to  achieve  this 
vocal  similarity  using  the  scheme  described 
above. 


The  experimenter's  capability  to  easily 
define  the  vocabulary  set  for  investigation  is 
provided  via  an  off-line  program  which  builds 
a disk  file  via  a dialog  with  the  user  on  the 
system  teletypewriter.  The  program  will  ac- 
cept the  specification  of  63  words  or  phrases 
in  each  file.  Each  of  these  63  items  is  de- 
fined by  giving: 

a.  The  expected  duration  of  the  phrase; 
e.g.,  800  milliseconds  for  "on  glidepath." 

This  datum  is  later  used  to  (optionally)  de- 
termine that  the  student  is  grossly  mispro- 
nouncing or  garbling  the  phrase. 

b.  An  alphanumeric  text  sequence  to  be 
associated  with  the  phrase.  These  data  may 
simply  represent  the  English  text  of  the 
phrase  or  a specially  defined  code  useful  to 
an  online  program. 

c.  A phonetic  text  sequence  to  be  asso- 
ciated with  the  phrase.  Again,  this  may  be 
the  English  pronunciation  of  the  phrase  or 
some  other  aural  cue  or  response  useful  in  the 
online  environment. 

d.  A syntax  structure.  Associated  with 
the  VDC  program  is  a rudimentary  syntactical 
processor  which  enables  the  concatenation  of 
phrases  within  the  vocabulary  list.  For  ex- 

r ample,  "assigned  heading"  must  be  followed  by 

three  digits.  This  specification  is  defined 
by  the  syntax  structure  associated  with  "as- 
signed heading.”  fable  1 briefly  describes 
the  syntax  options  available. 

In  addition  to  its  file  creation  capabil- 
ity, the  off-line  vocabulary  definition  pro- 
j gram  enables  the  experimenter  to  edit  the  disk 

file  and  generate  hardcopy  reports. 

VOCABULARY  PRESENTATION,  TRAINING. 

AND  VALIDATION 

Readers  familiar  with  the  GCA-CTS  efforts 
will  recall  that  a necessary  process  is  the 
extraction  of  voice  characteristics  of  each 
potential  user.  In  the  early  system,  computer 
core  restrictions  necessitated  a simple  serial 
presentation  of  each  individual  vocabulary 
item.  The  phrase  was  typed  onto  the  system 
teletypewriter,  and  the  student  was  prompted 
to  repeat  the  phrase  the  requisite  number  of 
times.  The  system  collected  data  on  the  stu- 
dent's vocal  characteristics,  generated  the 
necessary  reference  patterns,  and  moved  on  to 
the  next  vocabulary  item. 2 Unfortunately, 
however,  the  accuracy  of  the  recognition  func- 
tions of  the  speech  understanding  subsystem 
(SUS)  seemed  to  be  directly  related  to  the 
similarity  between  the  student's  voicing  of  a 
phrase  when  "configuring  the  recognition  sys- 
tem" and  during  regular  (GCA  controlling) 


Also,  during  the  early  GCA-CTS  efforts  it 
was  quickly  realized  that  this  "machine  train- 
ing" time  which  was  originally  required  only 
because  of  the  necessity  for  collecting  the 
speech  reference  data  - could  also  be  effec- 
tively itilized  as  "student  training"  time 
to  give  exposure  in  the  correct  pronunciation 
and  usage  of  the  GCA  vocabulary.  The  limita- 
tions of  serial  repetitions  and  individual 
phrases,  however,  severely  hampered  the  ex- 
ploitation of  this  dual  mode. 

In  the  VDC  program,  the  experimenter  is 
given  complete  flexibility  in  constructing 
(online  or  off-line)  sequential  presentations 
of  muHi phrase  combinations.  Additionally, 
the  experimenter  may  associate  a variety  of 
cues  with  these  phrases  via  the  alphanumeric 
and  phonetic  texts  in  the  vocabulary  defini- 
tion file.  Furthermore,  the  presentations 
can  be  done  independently,  in  combination  with 
the  reference  data  extraction  process,  or  in 
combination  with  the  recognition  processes. 
Taken  together,  these  capabilities  provide  the 
researcher  with  complete  freedom  in  devising 
a uniquely  tailored  program  which: 

a.  Allows  the  student  to  become  ac- 
quainted with  an  unfamiliar  vocabulary. 

b.  Transparently  collects  reference  data 
on  the  vocabulary  item  spoken  in  a realistic 
sequence. 

c.  Transparently  validates  the  accuracy 
of  the  reference  data  by  attempting  to  recoq- 
nize  the  phrase. 

In  short,  a system  can  be  quickly  and  easily 
developed  which,  from  the  student's  perspec- 
tive, is  teaching  him  when  and  how  to  speak 
the  vocabulary  phrases,  and,  from  the  experi- 
menter's perspective,  is  collecting  data  on 
the  speech  recognition  and  vocabulary  training 
functions.  A significant  improvement  in  the 
qua! ity  (accuracy)  of  the  extracted  speech 
reference  data  is  also  achieved. 

These  presentation,  training,  and  vali- 
dation functions  of  the  VDC  program  are  facil- 
itated by  an  easy  and  simple  interface  between 
the  researcher  and  the  software.  Disk  files 
and  the  presentation  devices  are  named  during 
a sign-on  dialog  with  the  system.  The  func- 
tional flow  is  directed  via  a conmand  file 
created  off-line,  or  by  entering  commands  on- 
line via  a named  input  device.  Examples  of 
these  functional  commands  (regardless  of  their 
source)  are  given  in  Table  2. 

If,  during  the  validation  mode  of  VDC 
program  operation,  the  user  does  not  specify 
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TABLE  1.  VDC  SYNTAX  OPTIONS 


4 


Cede 


Definition 


Examples* 


1 Complete  phrase 

2 Special  word^ 

3 One  of  N phrases  can  precede  this  phrase 

4 One  of  N phrases  can  follow  this  phrase 

5.  One  of  N phrases  must  procede  this  phrase 

6 One  of  N phrases  must  follow  this  phrase 

7 N special  words  can  precede  this  phrase 

8 N special  words  can  follow  this  phrase 

9 N special  words  must  precede  this  phrase 

10  N special  words  must  follow  this  phrase 


"exe<  uto  missed  appr oar  h" 

"zero,"  "one,"  "two,"  ...  "point," 
"oh" 

"above  gli depath  ...  coming  down"; 
"coming  down" 

"above  qlidepath  ...  coming  down"; 
"above  qTTdepath" 

"time  to  intercept  ...  track  bravo" ; 
"range  and  bearing  ...  track  bravo' 

“time  to  intercept  ...  track  bravo"; 
"time  to  intercept  ...  track  alpha" 

"7  ...  aircraft";  "aircraft" 

"turn  left  ...  0 ...  1 ...  0";  "turn 
left" 

"6  ...  miles  frqm_touchdown" ; 

“4  ...  miles  from  touchdown" 

"assigned  heading  ...  0 ...  1 ...  0"; 
"assigned  heading  ...  0 ...  1 ...  5" 


^Special  words  are  vocabulary  items  which: 

1.  may  stand  alone,  and/or 

2.  may  appear  in  a series,  and/or 

3.  may  precede  a phrase,  and/or 

4.  may  follow  a phrase. 


Phrases  being  defined  are  underscored. 
Other  phrases  provide  context  to  explain 
the  syntax. 


TABLE  2.  VDC  FUNCTIONAL  COMMANDS 


Mode  or  File 
Type 

Command 

No.  Arguments 

Example 

Meaning 

"Presentation 

P 

1 - 12 

"P  14" 

Present  phrase  14. 

and  Training" 

"P  13,  14,  15" 

Present  multiphrase  from  concatenated 
phrases  13,  14,  and  15. 

B 

1 

"B  0" 

Begin  collecting  voice  patterns  rot 
all  phrases. 

"B  10" 

Begin  collecting  voice  patterns  for 
phrase  10. 

S 

T 

"S  0" 

Stop  collecting  voice  patterns  for 
all  phrases. 

"S  10" 

Stop  collecting  voice  patterns  for 
phrase  10. 

T 

None 

"T" 

Terminate  presentation  function. 

"Validation" 

P 

As  above 

As  above 

As  above. 

A 

None 

"A" 

Print  all  statistical  data. 

S 

None 

“S" 

Print  summary  statistical  data. 

T 

As  above 

As  above 

Terminate  validation  function. 
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either  a command  file  or  other  input  device, 
the  system  will  (obviously)  not  present  phrases 
to  the  student  but  rather  will  "echo"  the 
phrases  which  it  recognizes.  The  syntax 
structure  defined  earlier,  together  with  some 
rather  sophisticated  logic,  is  used  to  con- 
catenate phrases  in  this  submode. 

RECOGNITION  ACCURACY  AND  DATA  COLLECTION 

An  important  aspect  of  the  VDC  program  - 
especially  when  considered  as  a laboratory 
investigation  tool  is  its  ability  to  easily 
and  quickly  assess  the  recognition  reliability 
of  the  hardware/software  system  and  identify 
misrecognized  or  confused  phrases.  Very  of- 
ten, once  these  risk  areas  are  delineated, 
the  eventual  training  system  can  be  designed 
and  implemented  to  support  the  recognition 
processes,*  and/or  minimize  the  impact  of  the 
recognition  mistakes. 

The  results  of  the  laboratory  evaluations 
of  the  GCA-CTS,  for  example,  have  indicated 
that  recognition  confusions  tend  to  occur 
between  similar  messages  for  the  computer  just 
as  they  do  for  humans,  as,  for  example,  the 
phrases  "slightly  above  glidepath"  and  "slight- 
ly below  glidepath."  For  some  talkers,  the 
plosive  "buh"  sound  in  both  phrases  tends  to 
obliterate  the  other  differences,  particularly 
if  the  above/below  portion  carries  unnatural 
emphasis,  and  thus  the  two  can  be  confused. 

But  consider  the  typical  glidepath  position 
error  made  by  a student  controller.  He  is  not 
likely  to  say  "slightly  above"  when  he  means 
slightly  below.  The  more  common  mistake  is 
failure  to  distinguish  between  adjoining  zones; 
for  example,  by  saying  "slightly  below"  when 
the  aircraft  is  actually  below.  In  the  latest 
GCA-CTS  programs,  delivered  in  May,  1976,  the 
Performance  Measurement  Subsystem,  in  com- 
munion with  the  SUS,  is  able  to  resolve  these 
recognition  errors,  resulting  in  understanding 
(as  opposed  to  recognition)  accuracies  ap- 
proaching 100  percent. 

The  real  point  to  be  made  here  is  that, 
when  typical  recognition  confusions  are  known, 
the  system  can  often  be  designed  to  tolerate 
them.  The  VDC  program  enables  the  researcher 
to  identify  these  problem  areas  very  early  in 
the  analysis  and  definition  phase,  resulting 
in  greater  visibility  going  into  the  design 
phase. 

Figure  1 demonstrates  a portion  of  the 
hardcopy  data  available  from  the  VDC  program. 
The  printout  includes  a header  which  provides 


♦Indeed,  this  is  one  of  the  distinctions 
between  a speech  understanding  system  and 
speech  recognition.  Understanding  utilizes 
contextual  and  other  cues  to  support  the 
recognition  algorithms. 


identifying  information  followed  by  each 
prompted  (presented)  phrase  and  by  the  recog- 
nized phrase.  Phrases  which  were  misrecog- 
nized are  flagged  with  an  "*."  Phrases  that 
are  recognized  with  low  confidence  are  noted 
with  an  "0."  Critical  recognition  parameters 
are  saved  for  reference,  and  two  overall 
phrase  accuracy  figures  are  calculated:  for 

all  phrases  and  excluding  low  confidence 
phrases.  If  requested  by  the  user,  the  pro- 
gram will  provide  more  detailed  "internal" 
data  collected  during  the  recognition  process. 
This  information  is  particularly  helpful  to 
determine  specifically  which  phrases  are  being 
confused,  the  degree  of  the  confusion,  and 
the  effects  of  the  various  recognition 
parameters . 

(In  the  "echo"  mode  described  earlier, 
the  comparative  data  shown  in  the  figure 
cannot  be  calculated  since  the  program  has  no 
a priori  information  on  which  phrases  the 
student  is  speaking;  i.e.,  no  phrases  are 
prompted.  The  VDC  program  will,  nevertheless, 
save  recognized  phrases  as  well  as  the  asso- 
ciated internal  data.) 

USE  OF  VDC  IN  THE  TOTAL  TRAINING  PROGRAM 

Though  primarily  a research  tool,  the 
VDC  program  is  capable  of  supporting  the  lab- 
oratory task  training  programs  as  well.  In- 
deed, the  VDC  concept  was  originally  intended 
as  a replacement  for  specific  elements  of  the 
earlier  GCA  training  system  software,  albeit 
with  increased  emphasis  on  vocabulary  inde- 
pendence. By  coupling  the  VDC  and  the  latest 
GCA-CTS,  together  with  the  DEC  display  group 
and  PDP-9  computer,  the  following  experi- 
mental training  scenario  has  been  achieved  for 
the  GCA  application. 

A new  student  is  signed  into  the  system. 
He  receives  an  audio/visual  presentation  of 
the  GCA  PAR  vocabulary  via  the  voice  genera- 
tion unit  and  graphics  display.  Via  "show- 
and-tell,"  the  student  is  instructed  in  t.ne 
proper  phrases  for  each  radar  pip  position. 

The  student  participates  in  a fol low-after- 
me  mode  in  which  he  repeats  GCA  commands  after 
an  appropriate  audio/visual  cue.  As  he  be- 
comes familiar  with  the  phraseology,  the 
instructor  quietly  informs  the  system  to  start 
collecting  voice  data  for  specified  phrases. 
The  student  continues  this  process,  always 
being  prompted  as  to  the  correct  phrase  for 
each  situation.  He  garbles  a phrase,  but  the 
system  forces  a repeat.  The  instructor, 
finally  satisfied  with  the  student's  progress, 
commands  the  system  to  generate  reference  data 
based  upon  the  information  collected  thus  far. 

The  student  returns  later  to  practice 
more  approaches.  At  first,  he  is  still 
prompted  by  the  system.  The  instructor  com- 
mands a report  on  recognition  accuracy. 
"Slightly  above  glidepath"  is  being  confused 
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VI)C  SUMMARY  DATA 


NAME:  GRADY,  MIKE  DATE:  5/31/76 

VOCAB  LIST  FILE:  VLG : GCACTS . VL  VOCAB  LIST  NAME:  GCA-CTS  VOCABULARY  REV  01 

VOICE  DATA  FILE:  DPI : GRADY .VD  VOICE  DATA  DATE:  5/31/76 


PHRASE  * : 1 

P:  SLIGHTLY  ABOVE  GLIDEPATH 
<®R:  SLIGHTLY  ABOVE  GLIDEPATH 


PHRASE  # : 2 

P:  SLIGHTLY  BELOW  GLIDEPATH 
R:  SLIGHTLY  BELOW  GLIDEPATH 


PHRASE  # : 3 

P:  GOING  FURTHER  ABOVE  GLIDEPATH 
R:  GOING  FURTHER  ABOVE  GLIDEPATH 


PHRASE  »:  4 

P:  GOING  FURTHER  BELOW  GLIDEPATH 
R:  GOING  FURTHER  BELOW  GLIDEPATH 


PHRASE  ♦ : 5 

* P:  FOUR  MILES  FROM  TOUCHDOWN 
®R:  TWO  MILES  FROM  TOUCHDOWN 


PHRASE  * : 6 

P:  TURN  LEFT  HEADING  ZERO  ONE  FOUR 
R:  TURN  LEFi  HEADING  ZERO  ONE  FOUR 


PHRASE  # : 7 

P:  WIND  1 1 0 A1  1 0 
R:  WIND  1 1 0 A1  1 0 


PHRASE  # : 8 

P:  WIND  1 1 0 A1  2 0 
R:  WIND  1 1 0 A1  2 0 


M = ISO  T = 60  VBCLS  = 6 VBTV  = 30  VCONF  = 50 

PERCENT  CORRECT  RECOGNITION : 87.5 

WHEN  LOW  CONFIDENCE  ITEMS  CAN  BE  RESOLVED, 

PERCENT  CORRECT  RECOGNITION:  100.0 


Figure  1.  VDC  Summary  Data  - With  Prompting 
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PILOT  ACCEPTANCE  AND  PERFORMANCE  EVALUATION  OF  VISUAL  SIMULATION 
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The  primary  value  of  the  effort  which  is  the 
subject  of  this  report  is  that  it  represents 
a novel  application  of  research  techniques 
usually  reserved  for  more  academic  pursuits 
to  simulation  system  procurement.  The  re- 
quirement for  this  test  was  specified  in  the 
document  provided  to  prospective  suppliers  of 
the  visual  simulation  system  sought  by  the 
Flight  Crew  Training  organization  of  the 
Boeing  Commercial  Airplane  Company  for  in- 
stallation on  its  moving  base  simulators  for 
the  707,  727,  737,  and  747.  The  more  common 
approach  appears  to  be  limited  to  reliance  on 
opinions  of  a few  senior  flying  personnel  and 
members  of  management.  The  present  study 
(while  not  permitting  the  rigor  attainable  in 
the  more  narrowly  circumscribed  laboratory 
experiment)  is  an  example  of  the  extension  of 
research  techniques  to  equipment  procurement 
involving  the  display  of  complex  perceptual 
stimul i . 

Without  a simulation  of  the  extra-cockpit 
visual  scene,  simulation  of  the  aircraft  is 
obviously  limited  to  instrument  flying  (IFR). 
In  the  "blind"  simulation,  the  simulator 
pilot  can  follow  his  approach  down  to  a hun- 
dred feet  or  so  before  he  must  consider  it 
completed.  The  value  of  this  type  of  train- 
ing cannot  be  questioned,  insofar  as  safety 
requires  IFR  skill  when  visibility  is  poor. 
However,  a critical  segment  is  left  out  of 
such  simulation;  a segment  requiring  precise 
coordinated  responses  to  the  outside  scene. 

The  last  few  feet  before  touchdown  require 
maneuvers  in  response  to  heads-up  visual  in- 
formation including  the  critical  flare  maneu- 
ver. Roll-out,  with  thrust  reversal  and 
braking,  obviously  entails  such  visual  input. 
Takeoff  is  another  maneuver  in  this  category 
as  is  the  somewhat  demanding  engine-out  pro- 
cedure after  rotation. 

The  savings  derived  from  moving  training  from 
the  aircraft  to  the  simulator  are  immediately 
apparent  in  that  the  aircraft  is  much  more 
expensive.  There  is,  moreover,  additional 
savings  in  training  time  because  the  simu- 
lator can  be  repositioned  in  x,  y,  and  z to 
some  desired  starting  point  without  taking 
the  time  (and  fuel)  to  fly  there.  Additional 
savings  derive  from  the  opportunity  to  inter- 
rupt the  flight  at  any  point  to  discuss  some 
particular  aspect  of  pilot  performance  needing 


attention.  Finally,  the  simulator  can  be 
used  for  training  in  emergency  maneuvers 
which  are  inherently  dangerous  to  attempt 
in  the  aircraft  and  under  poor  visibility. 

The  summary  in  Table  1 provides  a quick 
overview  of  some  major  advantages  and 
disadvantages  of  four  general  categories  of 
visual  simulation  techniques.  The  first 
two  are  computer-generated;  one  providing 
both  night  and  day  simulation,  while  the 
other  is  "night  only."  A third  type  uses 
movie  film  which  was  obtained  on  the  ap- 
proach to  the  airport  pictured.  The  fourth 
type  uses  a small  scale  terrain  model  (i.e., 
a solid  model  scaled  down)  an  optical  probe, 
and  a closed-circuit  television  (CCTV)  as 
its  major  components. 

All  four  types  of  visual  simulation  have 
inherent  weaknesses.  The  primary  limita- 
tions of  each  are  listed  at  the  end  of  the 
table.  The  six  individual  areas  of  concern 
are  listed  in  the  column  on  the  left. 

Realism.  The  aspect  of  visual  simulation 
to  which  most  observers  are  sensitive, 
initially,  is  that  of  realism.  Movie  film 
is  necessarily  most  realistic,  but,  because 
of  optical  limitations  and  film  wear,  the 
observer  is  occasionally  reminded  that  he 
is  looking  at  cinematography,  especially 
when  film  scratches  form  streaks  in  the 
field.  These  provide  incidental  cues  about 
glide  path  alignment.  The  rigid  model  with 
CCTV  is  generally  good  in  its  realism,  but 
is  limited  by  the  size  of  the  model  and 
by  the  scale  of  the  model  interacting  with 
limitations  of  the  optical  probe  and  the 
TV  display.  The  computer  generated  night- 
only  display  takes  advantage  of  the  fact 
that  points  of  light  are  easier  to  produce 
in  the  cathode-ray  tube  (CRT)  display  than 
surfaces  would  be.  The  computer  generated 
imagery  (CGI)  day/night  simulation  can 
appear  cartoon-like  because  of  the  require- 
ment to  limit  the  number  of  edges.  However, 
if  major  landmarks  and  a reasonable  amount 
of  other  details  are  programmed  into  the 
data  base,  the  dynamic  appearance  of  the 
scene  during  an  approach  imparts  a realism 
which  is  quite  compelling. 

Day/Night.  The  movie  film  display  is  limited 
by  the  photographic  and  projection  systems; 
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TABLE  1.  VISUAL  SIMULATION  SYSTEMS  COMPARISON 
(AS  OF  JUNE  1974) 


CGI  Day/Night 

Night  Only/CGI 

Rigid  Model 

Realism 

Dynamic 
rea 1 i sm 
good,  static 
picture 
cartoon-1 i ke. 

Good  but  limited 
to  points  of 
light. 

Excellent  (except 
see  Resolution). 

Good  but  limited 
by  size  of  model . 

Day/Night 

Yes,  also 
dusk. 

Night  only. 

Day  only. 

Day  only. 

Weather 

Variation 

Clouds , 
visibility, 
runway  visual 
range,  and 
scud. 

Clear,  obstruction 
by  clouds,  visual 
range  and  scud. 

Clear  or  with 
veiling  glare 
for  haze. 

Distortion 

Little  or 
none. 

Little  or  none. 

Can  be  severe 
when  off  local- 
izer/glide slope. 

Some  peripherally 
from  optical 
probe. 

Resolution 

Good  but 
short  of 
pilot  visual 
capability. 

Very  good  but 
short  of  pilot 
capability. 



Generally  poor. 

Poor  (at  display) 

Depth  of 
Field 

Not  limited. 

Not  limited. 

Camera  limited. 

Optical  probe  and 
model  scale 
1 imi ted. 

Primary 

Limitations 

Flicker. 

Night  only 
No  blue. 

Day  only 
Incidental 
cues,  streaking 
Lag  time. 

Poor  resolution 
and  depth  of 
field. 

Lag  time. 
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that  is,  the  sensitivity  of  the  film  and 
the  amount  of  light  lost  in  optical  systems 
to  provide  designed  orthographic  projection 
of  film  originally  photographed  from  a dif- 
ferent perspective.  Thus,  more  light  is 
required  in  the  original  scene.  The  rigid 
model  can  have  incorporated  points  of  light, 
but  a large  number  of  these  would  generally 
be  impractical  and  the  scale  of  the  model 
generally  makes  it  difficult  for  any  source 
to  be  seen  as  a point.  This  precludes  the 
presentation  of  a night  scene  with  this  type 
of  display.  The  day/night  CGI  can  also 
present  a dusk  scene  for  each  of  the  air- 
ports now  available  in  the  Boeing  data 
bases. 

Weather  Variation.  The  movie  film  of  ap- 
proaches must  of  necessity  be  limited  to  the 
weather  conditions  prevailing  at  the  time  of 
exposure.  Some  alteration  of  the  appearance 
of  the  scene  may  be  possible  by  "special 
effects"  techniques.  The  rigid  model  dis- 
play, for  example,  introduces  a veiling 
glare  into  the  scene  which  is  meant  to 
simulate  a haze  condition.  Other  than  this, 
nothing  was  available  in  these  two  systems 
at  the  time  of  the  review  to  simulate  weath- 
er variations.  The  night-only  simulation 
system  was  also  limited  to  a clear  weather 
condition  at  the  time  of  this  review.  Only 
the  CGI  day/night  provides  for  variable 
altitude  cloud  tops  and  bottoms  or  variable 
runway  visual  range  (RVR). 

Distortion.  There  is  little  distortion  in 
either  the  night/day  CGI  or  the  night-only 
system  if  the  CRT  is  adjusted  properly.  The 
rigid  model  system  may  show  some  distortion 
in  the  periphery  of  the  display  field  because 
of  limitations  in  the  optical  probe.  The 
most  severe  distortion  occurs  with  the  movie 
film  and  orthographic  projection  system  when 
the  simulator  is  off  glide  path.  Thus,  tall 
buildings  will  appear  to  tilt  away  from 
vertical  in  a distracting  way  with  film  dis- 
plays of  this  type  unless  the  projection 
coincides  with  the  original  viewing  angle. 

Resolution.  This  limitation  is  one  that 
affects  the  quality  of  all  the  displays, 
though  in  differing  amounts.  None  could 
match  the  resolution  (acuity)  of  the  human 
eye  which,  according  to  clinical  refraction 
standards,  is  nominally  one  arc  minufe  sub- 
tended at  the  eye.  This  visual  angle  is 
termed  Snellen  acuity  after  the  researcher 
who  devised  a set  of  letters  with  overall 
dimensions  of  five  units  and  details  (lines, 
gaps)  of  one  unit.  The  "average  normal"  or 
emnetrope  can  name  the  letter  when  its  de- 
tails subtend  one  minute  of  visual  angle. 

The  test  of  visual  acuity  made  with  such 
targets  obviously  requires  some  minimum 
level  of  literacy  and  could  not  be  admin- 
istered to  someone  who  did  not  know  the 
alphabet.  Furthermore,  many  have  finer 


acuity  than  the  one  minute  standard,  beinq 
able  to  discriminate  details  of  45  arc  sec- 
onds or  even  30  arc  seconds  in  the  Snellen 
letters.  Students  of  vision  are  also  aware 
that  the  visual  angle  for  normals  when  other 
types  of  test  targets  are  used  may  be  smaller 
or  larger  than  this  nominal  one  arc  minute 
for  legibility  of  letters.  For  example, 
vernier  acuity  (as  in  reading  the  minimum 
perceptible  offset  on  a slide  rule  or  vernier 
micrometer  scale)  is  about  five  arc  seconds, 
and  minimum  visible  acuity  (single  black  line 
against  a bright  background)  is  down  as  low 
as  0.5  arc  seconds.  Finally,  when  it  comes 
to  toe  minimum  visible  point  of  light  (e.g., 
a star)  it  can  be  almost  infinitely  small  as 
long  as  there  is  sufficient  light  in  it. 
However,  the  star  source  is  not  infinitely 
small  as  projected  on  the  retina  because  of 
optical  limitations  of  the  eye. 

The  importance  of  the  foregoing  discussion  is 
that  the  selection  of  a single  value  for  ac- 
ceptable resolution  in  a display  such  as 
those  being  considered  here  would  of  necessity 
be  arbitrary  in  some  resDect  or  another. 
Moreover,  the  adequacy  or  meaning  of  any 
stimulus  depends  upon  what  responses  you  must 
make  to  it  in  order  to  reach  some  desired 
goal.  As  a way  of  stating  in  general  terms 
our  minimum  requirements  for  visual  simu- 
lation for  flight,  we  ask,  "Can  the  pilot  fly 
to  the  display?"  Stated  another  way,  "Is  the 
visual  information  sufficient  for  the  pilot 
to  learn  what  he  must  for  later  successful 
performance  in  the  airplane?"  Considering 
resolution,  or  any  other  characteristic  of 
the  visual  simulation,  in  isolation  will 
not  yield  an  answer  to  this  auestion.  This 
is  the  primary  reason  for  research  of  the 
type  reported  here. 

Resolution  in  the  movie  film  display  is 
generally  poor,  primarily  because  of  optical 
considerations.  One  of  the  potential  virtues 
of  photography,  that  of  using  the  real  scene 
with  all  its  details,  is  actually  & detriment 
to  its  acceptability.  Such  detail  in  the 
visual  world  provides  texture  whicn  enhances 
the  appearance  of  depth.  Lack  of  resolution 
in  its  photographic  representation  thus 
becomes  more  noticeable  by  the  impoverish- 
ment of  this  texture.  The  terrain  board 
(rigid  model)  also  loses  fine  detail  as  the 
image  goes  through  optics  and  TV  display 
components.  The  blur  is  quite  noticeable, 
but  is  less  distracting  than  that  associated 
with  movie  film,  partly  because  the  terrain 
model  scene  is  not  as  complex  as  the  visual 
world  would  be  in  the  airplane.  The  night- 
only  system  need  only  worry  about  the  size 
of  its  point  lights,  and  these  were  good  in 
the  displays  available  to  view.  The  CGI  day/ 
night  display  achieved  at  best  three  arc 
minutes;  while  this  was  as  good  as  the  night 
only,  it  still  was  too  larqe  to  match  human 
visual  capability. 


Depth  of  field  The  day/night  and  night- 
only  systems  have  no  depth  of  field  problem, 
because  there  are  no  optical  components  used 
to  bring  the  image  to  the  display.  Cinema- 
tography uses  optics  but,  because  of  the 
distaries  involved,  all  details  outside  the 
cockp't  are  beyond  infinity  focus  for  the 
camera.  The  rigid  model  system,  however, 
does  have  a problem  associated  with  the 
foreshortened  distances  in  a small  scale 
model.  Point  lights  (e.g.,  runway  lights) 
tend  to  balloon  out  as  they  are  approached 
In  the  simulation,  and  surface  details  be- 
come blurred.  One  partial  solution  to  this 
problem  is  to  change  over  to  a terrain  model 
of  larger  scale  (representing  a smaller 
ground  area)  with  correspondingly  larger 
details.  Such  an  approach  obviously  compli- 
cates the  system  and  introduces  undesirable 
discontinuities. 

Primary  Limitations.  Each  of  the  four  gen- 
eral types  of  visual  simulation  systems  has 
unique  advantages  and  unique  disadvantages, 
a few  of  which  have  been  covered  in  the  pre- 
ceding discussion.  This  brief  treatment 
will  touch  on  the  limitations  which  appear 
most  specifically  related  to  each  simulation 
technique.  The  CGI  day/night  system  has,  in 
the  day  scenes,  a tendency  to  flicker  as  the 
detail  begins  to  correspond  to  one  raster 
line.  For  example,  if  runway  markings  are 
displayed  from  several  miles  distance,  they 
appear  to  shimmer  and  flicker.  Fortunately, 
this  kind  of  problem  can  be  overcome  by 
waiting  until  the  aircraft  is  closer  to  the 
runway  before  introducing  such  details  into 
the  scene.  Some  flicker  remains  neverthe- 
less, and  the  removal  of  this  nega^'ve 
feature  of  CGI  day  scenes  awaits  future 
improvements  in  the  technique. 

The  night-only  scene  has,  in  addition  to  the 
restriction  to  night  scenes,  no  blue  light 
in  the  scene.  In  cases  where  this  color 
may  be  desired,  for  example  in  runway  edge 
lighting,  it  is  simply  unavailable  in  the 
present  night-only  simulation. 

The  movie  film  disadvantages  lie  primarily 
in  the  incidental  cues  to  performance  error. 
Distortions  arising  from  the  anamorphic  pro- 
jection system  could  provide  a clue  to  glide 
path  deviation  which  the  student  pilot 
might  learn  to  use  instead  of  the  appro- 
priate cues.  A more  disturbing  artifact 
observable  in  cinematographic  systems  of 
visual  flight  simulation  is  the  streaking 
which  appears  in  the  display  from  scratches 
on  the  film  after  repeated  use.  Flying  to 
the  streaks  instead  of  to  the  projected 
scene  would  be  somewhat  difficult  for  the 
student  to  avoid,  especially  when  he  is 
anxious  to  "look  good"  to  the  one  who  is 
evaluating  his  progress. 

The  major  limitations  to  the  use  of  terrain 


models  with  a "flying  optical  probe"  and  a 
CCTV  have  already  been  discussed  above.  They 
are  poor  resolution  in  the  display  and  re- 
stricted depth  of  field,  the  latter  causing 
nearby  details  to  be  blurred.  A further  con- 
sideration which  causes  hesitation  on  the 
part  of  prospective  buyers  of  visual  flight 
simulators  is  that  a not  insignificant  volume 
of  space  is  required  for  the  terrain  models 
themselves. 

Evaluation  Criteria.  The  foregoing  discus- 
sion has  been  concerned  primarily  with  the 
purposes  for  which  visual  simulators  are 
purchased  and  the  good  and  bad  points  of  the 
various  general  techniques  of  visual  simu- 
lation. It  was  suggested  in  the  introduction 
that  the  important  contribution  of  the  effort 
herein  reported  is  the  use  of  the  research 
techniques  of  the  experimental  psychologist 
as  a method  of  ensuring  compliance  by  the 
supplier  with  specifications  the  user  would 
like  to  impose  on  system  performance. 

Uppermost  in  the  researcher's  mind,  of  course, 
is  the  quantitative  data  through  which  he 
asserts  his  faith  in  the  scientific  method. 

In  exploratory  studies  like  this  one,  many 
measures  must  be  examined  in  the  search  for 
the  best  indices  of  pilot  performance. 

In  addition  to  the  adequacy  of  the  design  of 
the  simulator  as  an  aid  in  the  training  of 
flight  crews  from  the  standpoint  of  essential 
elements  for  meeting  specific  behavioral 
objectives,  its  acceptability  in  subjective 
(perhaps  we  should  say  "esthetic")  terms  is 
important.  If  it  "looks  funny"  to  the  student, 
the  visual  simulation  may  be  ineffectual  for 
its  major  purpose.  Therefore,  it  was  desir- 
able to  get  the  instructor  pilots  who  flew 
the  simulator  in  our  evaluation  to  offer 
their  opinions  about  the  acceptabi 1 ity  of 
the  visual  simulation.  We  were  also  inter- 
ested in  the  reactions  of  these  instructor 
pilots  to  the  potential  for  improved  flight 
crew  training  provided  by  the  addition  of 
the  visual  simulation  to  our  moving-base 
simulators . 

The  ultimate  proof  of  the  value  of  any  train- 
ing aid  such  as  a simulator  is  the  degree  to 
which  the  learning  later  on  transfers  to  the 
operational  task.  Although  this  type  of 
evaluation  is  part  of  the  Boeing  Flight  Crew 
Training  visual  simulation  test  program,  it 
was  not  included  in  the  procurement  specifi- 
cations because  the  magnitude  of  such  a test 
would  impose  a delay  in  acceptance  which 
would  have  been  unfair  to  the  supplier. 

Visual  Simulator  Evaluation 

Test  Facility.  All  quantitative  data  was 
obtained  from  simulator  readouts  as  the 
pilots  made  their  approaches  and  landings 
in  the  Boeing  737  and  727  Flight  Crew 
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Training  simulator  at  Boeing  Field. 

Pilots.  The  data  was  provided  by  16  Boeing 
instructor  pilots  currently  active  in  that 
assignment . 

Data  Base.  The  scene  consisted  of  the  air- 
port and  environs  at  Moses  Lake,  Washington 
(Grant  County  Airport).  The  runway  is 
13,500  ft.  by  400  ft. 

Flights.  Each  pilot  made  eight  approach/ 
landings. 

Experimental  Conditions.  Three  variables 
were  treated  in  the  test.  The  first  was 
the  use  of  instrumentation.  While  all 
flights  were  made  without  altimeter  visible, 
on  half  the  flights  (4)  the  pilot  was  not 
given  glide  slope/localizer  information 
either.  The  flights  were  evenly  divided 
between  day  and  night  approaches  (4  each). 

The  third  variable  concerned  visibility. 

Under  the  good  visibility  condition,  the  run- 
way visual  range  was  200,000  ft.  and  visi- 
bility was  30  nautical  miles,  while  the 
limited  condition  consisted  of  a runway 
visual  range  of  36,000  ft.  and  visibility 
of  20  nautical  miles.  As  with  the  other 
two  variables,  the  two  visibility  conditions 
were  evenly  divided,  4 and  4.  All  three 
variables  are  combined  in  counterbalanced 
fashioned  to  ensure  that  interaction  pairings 
are  evenly  distributed.  The  actual  presen- 
tation order  was  mixed  up  (semi-randomized) 
to  keep  order  effects  to  a minimum.  (Fig.  1) 

Approach  Path.  The  simulator  was  positioned 
at  1330  ft.  above  the  runway  level  and  4.7 
nautical  miles  out,  on  glide  slope  (2.5°)  at 
the  outer  marker.  There  was  a wind  coming 
from  the  right  at  20  knots.  This  wind  drop- 
ped to  10  knots  as  the  aircraft  passed  over 
the  middle  marker  (0.5  n.mi.)  and  dropped 
further  to  zero  at  50  ft.  above  the  runway. 

The  touchdown  point  was  0.3  n.mi.  down  the 
runway  beyond  the  threshold. 

Performance  Measures.  At  intervals  of  one 
second,  seven  items  of  information  were  print- 
ed out.  They  were  1)  time  from  start  of 
descent,  2)  ground  distance,  3)  groundspeed, 

4)  altitude,  5)  distance  left  or  right  from 
localizer,  6)  rate  of  descent,  and  7)  touch- 
down point  (wheels  are  touching).  These 
measures  were  chosen  as  the  ones  most  likely 
to  include  valid  indices  of  performance. 

When  sufficient  data  are  amassed  they  hope- 
fully will  permit  us  to  select  a couple  of 
predictors  which  will  account  for  most  of 
the  variation  in  performance  among  pilots. 

Procedure.  Prior  to  entering  the  simulator 
cab,  the  pilot  was  given  an  instruction  as  to 
the  nature  of  the  task  and  the  purpose  of 
the  test.  In  addition  to  the  strictly  infor- 
mational content  of  the  Instruction,  an 


emphasis  was  placed  on  the  evaluation  of  a 
visual  display  as  opposed  to  an  evaluation 
of  their  flying  skills.  However,  they  ap- 
peared to  react  to  their  participation  such 
that  they  could  be  expected  to  do  their  best. 

Training.  To  ensure  at  least  a minimum  level 
of  familiarity  with  the  characteristics  of 
the  simulator  and  the  visual  display,  four 
practice  trials  were  run  before  the  eight  test 
trials  were  initiated.  In  the  first  of  these 
practice  trials, the  simulator  was  "flown" 
with  all  instruments  available  but  without  a 
visual  display.  The  starting  altitude  was 
1330  ft.  and  the  distance  from  the  glide 
slope  touchdown  point  was  30,400  ft.  The 
runway  was  400  ft.  wide  and  13,500  ft.  long 
and  the  glide  slope  intersection  with  the 
runway  was  1840  ft.  beyond  threshold. 

The  second  practice  approach  was  the  same  as 
the  first  except  that  the  daylight,  high 
visibility  scene  was  present  in  front  and 
side  windows.  The  pilot  was  reminded  to  make 
his  touchdown  at  the  same  point  (glide  slope  1 
runway  intercept)  instead  of  a point  closer 
to  runway  threshold  following  the  usual  prac- 
tice with  good  VFR  conditions. 

The  third  practice  approach  differed  from 
the  second  only  in  that  it  used  a night  scene. 
The  fourth  and  final  practice  aoproach  was 
identical  to  the  first,  with  no  visual  simu- 
lation but  with  all  instruments  available. 

Data  were  taken  on  the  practice  runs, but  they 
were  not  used  for  the  main  test  analysis. 

The  analysis  of  the  practice  data  was  under- 
taken separately  for  the  purpose  of  determin- 
ing the  effect  upon  performance  of  first  ex- 
posure to  the  visual  scene  (see  Appendix  B). 

Presentation  Order.  To  minimize  the  possi- 
bility that  variation  in  performance  across 
conditions  could  be  interpreted  in  terms  of 
the  order  in  which  they  were  presented, 
partial  randomization  was  used  for  the  eight 
test  runs.  For  each  combination  of  time 
day  and  visibility  condition,  both  instrument 
conditions  (with  and  without  glide  slope) 
were  run  before  proceeding  to  the  next  such 
combination.  This  was  deemed  adequate  to 
obviate  concerns  about  order  effects  in  the 
data  and  it  had  the  advantages  of  reducing 
the  opportunity  for  errors  in  setting  up 
conditions  and  reducing  the  time  needed  for 
the  completion  of  a session. 

Questionnaire.  After  all  test  runs  were 
completed,  each  pilot  was  queried  about  cer- 
tain aspects  of  the  simulation  (see  Appendix 
A).  The  questions  referred  to  such  things 
as  realism  in  the  visual  simulation,  potential 
improvements  in  training  arising  from  the 
addition  of  visual  simulation,  its  limi- 
tations and  the  quality  of  its  interaction 
with  motion  simulation.  After  a final 
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Figure  1.  The  Plan  for  Performance  Evaluation 


question  about  his  flying  experience,  the 
pilot  was  encouraged  to  comment  freely  about 
any  aspect  of  the  simulation  or  test  he 
wished.  The  comments  made  in  response  to  the 
questionnaire  and  those  made  freely  before  or 
after  it,  were  recorded  on  tape  for  later 
transcription. 

Results^  Figure  2 shows  the  average  alti- 
tude along  the  approach  path.  The  path 
flown  by  the  pilots  when  they  had  no  glide 
slope  information  tended  to  average  slightly 
lower  than  when  this  information  was  avail- 
able. The  raw  data  show  this  deviation  from 
glide  slope  (glide  slope  is  the  straight  line 
designated  "theoretical"  in  the  figure)  to  be 
due  primarily  to  performance  in  the  approach 
to  the  night  scene  with  good  visibility  but 
without  an  altimeter.  This  observation  is 
in  agreement  with  a similar  one  made  in 
prior  research  by  the  present  authors.  It 
was  noted  that  pilots  who  were  skilled 
enough  to  be  flight  crew  instructors  tended 
to  fly  low  when  moving  over  dark  terrain  on 
a night  visual  approach  when  they  lacked 
altitude  information.  Upon  maxing  this 
observation  in  the  prior  study,  we  hypothe- 
sized that  the  pilots  were  choosing  a path 
which  appeared  to  them  to  be  straight,  but 
which  in  fact  was  curved  because  it  was 
based  on  the  maintenance  of  a constant 
visual  angle  subtended  at  the  eye  by  the 
group  of  lights  in  the  scene.  This  path 
is  indicated  by  the  dashed  line  in  Figure 
2.  Inside  of  two  miles  the  path  flown 
by  the  pilots  in  this  study  appears  to  be 
a compromise  between  the  theoretically 
perfect  straight  line  glide  slope  and  the 
equal  visual  angle  path. 

Statistical  analyses  were  performed  on  the 
digital  data  which  was  read  out  at  one 
second  intervals.  In  the  analysis  of 
variance,  the  results  showed  three  statis- 
tically significant  (p  < .05)  effects. 

These  are  the  starred  means  in  Table  2. 

Two  of  the  effects  do  not  appear  to  have 
much  practical  significance:  A slightly 

higher  average  airspeed  when  visibility 
was  lower  and  an  average  of  one  second 
longer  before  touching  down  when  the  glide 
slope  inf  inflation  is  added  to  the  visual 
scene.  T ie  increase  in  airspeed  with  lower 
visibility  was  only  1.4  and  the  increased 
time  with  glide  slope  information  was  only 
0.7  , hardly  an  observation  worthy  of  much 
attention  with  our  present  level  of  sophisti- 
cation in  making  such  measures. 

The  third  effect  relates  to  the  distance 
from  wt-Vre  the  glide  slope  intersects  with 
the  runway  to  the  touchdown  point.  This 
distance  (the  X dimension)  is  recorded  in 
feet,  positive  in  sign  until  the  glide  slope/ 
runway  intercept  is  passed  where  it  becomes 
negative.  When  the  glide  slope  information 
was  present,  'andings  tended  to  be  "long” 


with  TD  point  a little  beyond  point  zero. 
Touchdown  without  glide  slope  information 
available  was  short  of  point  zero  by  over 
200  ft.  on  the  average. 

It  might  be  interpreted  that  the  use  of  glide 
slope  leads  to  better  approach  and  landing 
performance  on  the  basis  of  the  differerice 
observed  between  the  two  conditions.  However, 
it  may  be  that  these  pilots  were  merely  land- 
ing in  a manner  which  is  good  practice  in  the 
airplane  regardless  of  the  instruction  to 
touchdown  at  the  glide  si  ope/ runway  intercept 
point.  Certainly,  it  is  good  practice  in 
general  to  set  the  airplane  down  as  soon  as 
possible  (with  some  reasonable  margin)  in 
order  to  have  the  maximum  available  distance 
for  rollout.  However,  there  are  at  least 
two  other  hypotheses  which  could  also  account 
for  this  result.  The  display  was  rather  bare 
of  ground  lights  between  one  and  two  miles 
from  runway  threshold,  and  this  may  have  made 
altitude  judgment  more  difficult  to  do 
visually.  Also,  the  points  of  light  were 
never  as  small  as  the  theoretical  limit  of 
1.25  arc  minutes  of  visual  resolution  for 
a point  of  light;  all  were  larger  than  3 arc 
minutes.  There  is  no  way  to  determine  the 
correct  exaplanation  in  the  absence  of  the 
necessary  data.  Finally,  with  an  airspeed  of 
216  ft.  per  second  the  observed  difference 
cannot  be  said  to  be  anything  but  small  re- 
gardless of  its  statistical  significance. 

These  tests  were  based  on  the  data  recorded 
in  the  interval  just  before  touchdown  and 
the  criterion  measure  was  the  distance  between 
touchdown  point  and  the  intersection  of  the 
glide  slope  with  the  runway. 

Rather  than  use  the  obvious  measure  of  rms 
deviation  from  glide  slope  and  lateral  dis- 
tance from  runway  centerline  as  the  measure 
of  performance  along  the  entire  approach 
path,  the  data  recorded  just  before  passing 
over  the  middle  marker  was  used  in  the  second 
analysis  of  performance.  In  addition  *o  the 
problem  of  time  and  cost,  there  was  the 
question  of  selecting  the  appropriate  sen- 
sitive criterion  measure.  The  data  showed  an 
initial  drift  off  localizer  at  the  start  of 
the  descent  which  appeared  to  be  in  excess 
of  what  the  pilot  could  compensate  for.  When 
questioned  about  it,  pilots  indicated  that 
they  deliberately  tested  the  strength  of 
the  side  wind  by  letting  it  blow  them  off 
course  for  a period  of  time  before  trying  to 
correct  for  it.  This  appeared  to  be  an 
efficient  way  to  establish  the  crab  angle 
necessary  to  maintain  the  aircraft  on  local- 
izer for  the  remainder  of  the  descent  path. 
Therefore,  because  of  the  individualistic 
approach  adopted  by  the  pilots  (some  allowed 
a little  drift,  others  a lot),  rms  flight 
path  deviation  seemed  inappropriate  as  the 
measure  of  quality  of  performance.  There 
are  many  deviations  from  a straight-in 
approach  on  glide  slope  and  localizer  which 
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Figure  2.  Approach  Altitude  as  a Function  of  Distance  Computer 
Generated  Image  Evaluation 


TABLE  2.  737  CGI 

EVALUATION  ANOVA 

CELL  MEANS 

TOUCHDOWN 

TIME 

OF  DAY 

VISIBILITY 

DISPLAY 

DAY 

NIGHT 

GOOD 

LOW 

VISION 

ONLY 

GL  SLOPE 
* VISION 

X 

83.5 

105.1 

91.3 

97.4 

243.0* 

-54.4 

y 

7.7 

11.9 

14.1 

5.4 

7.3 

12.2 

Z 

-5.88 

-4.42 

-4.76 

-5.54 

-4.59 

-5.72 

Airspeed 

218. 

215 

215 

218* 

216 

216 

T ime 

137 

138 

138 

137 

137 

138* 

* P .05 
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would  not  cause  the  pilot  making  them  to  be 
downgraded  in  his  performance  evaluation; 
a criterion  measure  which  did  not  take  this 
into  account  might  well  be  considered  un- 
realistic. 

Meeting  a designated  glide  slope  touchdown 
point  is  also  somewhat  arbitrary  in  that  the 
II.S  is  meant  to  be  used  only  to  bring  the 
aircraft  in  sight  of  the  runway,  not  to  per- 
mit "blind"  landings.  Flaring  the  aircraft 
too  soon  or  too  late  is  a recognized  error, 
although  there  is  certainly  a range  of 
acceptable  flare  points.  Future  research 
in  the  evaluation  of  the  quality  of  simu- 
lation for  flight  crew  training  must  empha- 
size the  importance  of  the  appropriate 
criterion  measures. 

For  the  purposes  of  the  test  reported  here, 
a rough  measure  was  adequate  in  that  the 
major  concern  was  to  demonstrate  the  ade- 
quacy of  the  CGI  visual  simulation.  For 
tests  demanding  greater  sensitivity  (i.e., 
designed  to  show  the  existence  of  smaller 
differences  in  pilot  performance)  some  other, 
perhaps  less  obvious  measures  should  be 
developed.  We  chose  the  glide  slope  devi- 
ation at  the  middle  marker  as  being  sensi- 
tive to  the  quality  of  pilot  performance 
without  appearing  too  arbitrary  a measure. 

The  rationale  was  that  at  a point  just 
prior  to  passing  the  middle  marker,  where 
the  crosswind  dropped  from  20  knots  to  10 
knots,  the  pilot  had  had  sufficient  time 
under  constant  conditions  to  get  on 
localizer  and  at  the  proper  altitude  to 
make  good  touchdown  about  15  seconds  later. 

The  data  were  taken  from  a point  just  be- 
yond 0.6  n.mi.  from  runway  threshold. 

These  data  (Table  3)  showed  that  the  mean 
altitude  for  the  approaches  made  without 
glide  slope  information  was  25  ft.  rower 
than  the  corresponding  altitude  for  ap- 
proaches made  with  glide  slope  deviation 
displayed.  This  was  calculated  as  statis- 
tically significant,  with  a probability 
of  less  than  5%  of  being  attributable  to 
chance  variation. 

The  effect  of  visibility  on  altitude  appears 
to  be  limited  to  night  approaches  ( p < . 05 ) 
with  good  visibility  leading  to  lower  alti- 
tudes on  the  average.  The  common  sense 
explanation  for  this  would  appear  to  be 
that  a higher  approach  provides  a margin  of 
safety  for  the  lowered  visibility  situation. 
This,  of  course,  is  mere  speculation  since 
the  test  was  not  designed  for  such  a 
hypothesis. 

Although  the  difference  in  airspeed  of  2.5 
knots  between  "good"  and  "intermediate" 
visibility  levels  is  statistically  signifi- 
cant (p  < .01),  guessing  at  the  probable 
cause  for  the  difference  is  difficult  unless 


one  resorts  to  some  supposedly  correlated  ob- 
servation, such  as  the  relationship  between 
altitude  and  descent  rate  to  air  speed.  Thus, 
in  line  with  such  an  argument,  air  speed  would 
be  greater  at  the  higher  altitudes  and 
higher  rates  of  descent  (although  the  latter 
was  not  significant  at  p < .05  but  only  be- 
tween this  level  and  p < .10).  This  manner 
of  reasoning,  while  speculative,  is  indi- 
cative of  the  necessity  to  treat  performance 
realistically  in  terms  of  its  inherent  com- 
plexity rather  than  to  select  in  arbitrary 
fashion  something  which  appears  to  be  easy 
to  measure.  Any  instructor  pilot  knows  that 
he  must  evaluate  the  quality  of  student  pilot 
performance  in  terms  of  a four-dimensional 
envelope  involving  where  he  is  at  a given 
point  in  time  (three  dimensions)  and  how  he 
got  there  (4th  dimension  or  time).  There 
is  not  only  some  latitude  which  represents 
acceptable  performance  but  the  dynamic  aspect 
must  also  be  considered  in  its  assessment. 

An  example  could  be  given  from  the  present 
evaluation.  Pilots  were  told  that  the  pri- 
mary criterion  measure  would  be  the  distance 
from  runway  centerline/glide  slope  inter- 
cept to  the  point  where  they  actually 
touched  down.  One  pilot  set  down  somewhat 
harder  than  good  practice  would  dictate, 
presumably  in  conformity  with  what  had 
been  requested  of  him,  when  touching  down 
a little  farther  down  the  runway  would  have 
been  much  better  in  the  airplane  itself. 

Fie  was  then  told  that  hitting  the  target 
TD  point  was  not  to  be  accomplished  in 
violation  of  other  considerations  con- 
cerning good  flying  practice. 

The  last  item  listed  in  Table  3 on  per- 
formance at  0.6  n.mi.  out  is  azimuth  or 
distance  from  centerline  alignment.  Althouqh 
the  means  for  both  good  and  intermediate  visi- 
bility conditions  are  close  to  centerline, 
there  is  a statistically  significant  dif- 
ference (p  < .05)  between  the  two  with  the 
"intermediate"  slightly  further  left  of 
centerline  than  the  "good"  condition.  The 
most  likely  assumption  as  to  the  cause  of 
this  difference  would  appear  to  be  that 
maintaining  the  proper  crab  angle  with  a 
20  knot  crosswind  is  more  difficult  with 
reduced  visibility. 

Conclusions.  This  test  has  demonstrated 
conclusively  that  the  computer  generated 
image  display  provides  a visual  simulation 
which  is  acceptable  to  crew  training  per- 
sonnel (eight  instructor  pilots  in  the 
737  and  another  eight  in  the  727  simulator) 
in  terms  of  its  probable  utility  for  im- 
proved efficiency  in  training.  It  can  be 
"flown"  to  touchdown  without  altitude  or 
glide  slope  information  using  only  the 
visual  feedback  from  the  displayed  imaqery. 

The  test  yielded  data  which  were  stable 
enough  to  show  small  differences  as  being 
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TABLE  3.  CGI  EVALUATION 

PILOT  PERFORMANCE  WITH  737  SIMULATOR 
ANALYSIS  AT  0.0  NM  FROM  GLIDE  SLOPE  ORIGIN 
JUST  BEFORE  MIDDLE  MARKER  AND  WIND  SHEAR  (20  TO  10K) 


DISPLAY 

ALTITUDE 


VISION  ONLY  VISION  + G.S. 


210.9  FT 


235.4  FT 


P < 


TIME  O^  DAY  X VISIBILITY  VISIBILITY 

GOOD  INTERMEDIATI 


ALTITUDE 


DAY 

NIGHT 


224.5  FT 

220.2  FT 

209.8  FT 

233.1  FT 

P < 


VISIBILITY  GOOD  INTERMEDIATE 

SPEED 

TIME 

DESCENT  RATE 
AZIMUTH 


127.9  K 

130.4  K 

| 1.88  MIN 

1.806  MIN 

9.05 

10.29 

55.1 

68.8 

P <. 
P <■ 
P <. 
P <. 


.05* 

.05* 

■01“ 

.10 

.10 

.05* 
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statistically  significant. 

All  of  the  instructor  pilots  participating  in 
the  test  felt  that  instruction  would  take  ^ss 
time  and  require  less  time  in  the  aircraft 
itself.  An  example  of  the  benefits  they  felt 
would  be  derived  from  the  use  of  a day/night 
visual  simulation  is  the  transitioning  from 
one  aircraft  type  to  another.  The  visual 
scene  through  the  front  window  on  the  ap- 
proach is  an  important  cue  for  maintaining 
proper  aircraft  attitude  on  the  approach. 

This  is  especially  important  in  transition- 
ing from  propeller-driven  airplanes  to  jets, 
where  the  pitch  attitude  is  nose-high  on 
the  approach. 

The  most  valuable  outcome  of  the  test 
(beyond  acceptance  of  the  visual  simulator) 
regarding  future  potential  for  the  simu- 
lator is  related  to  its  apparent  effective- 
ness for  the  discrimination  of  fine  differ- 
ences in  pilot  performance.  All  of  the  land- 
ings made  in  the  simulator  during  the  test 
were  good  landings  as  cne  would  expect  with 
instructor  pilots. 

In  Table  2,  for  example,  where  "visual  plus 
glide  slope"  is  compared  with  "visual"  alone, 
a difference  in  speed  of  only  2.5  knots  is 
determined  to  be  statistically  significant 
(p  < . 05 ) . Although,  as  mentioned  previously, 
the  practical  significance  is  not  immediately 
apparent,  the  implied  stability  of  this  mea- 
sure is  encouraging  in  its  unexpected  sen- 
sitivity. Instead  of  relying  solely  on 
gross  "go,  no  go"  subjective  evaluations 
of  pilot  performance,  flight  crew  training 
personnel  may  be  able  to  use  more  subtle 
indicators  which  are  also  more  objective. 

The  discovery  of  such  indicators  requires 
serious  research  efforts,  but  the  avail- 
ability of  good  simulator  facilities  with 
appropriate  recording  equipment  would  be 
required.  It  would  not  be  feasible  eco- 
nomically to  attempt  to  develop  such  mea- 
sures in  the  aircraft. 

In  addition  to  their  potential  value  for 
finer  measures  of  pilot  performance,  the 
development  of  such  indicators  may  be  of 
direct  benefit  in  the  training  process 
itself.  Complex  behaviors  such  as  those 
involved  in  the  piloting  of  aircraft  must 
in  part  be  learned  by  a "shaping"  process. 
Instead  of  instructing  the  student  as  to  the 
precise  manner  in  which  an  activity  is  to 
be  carried  out,  the  instructor  tells  the 
student  the  desired  results  and  rewards  him 
when  he  is  able  to  achieve  it.  The  student 
learns  "how  it  feels"  when  performing 
correctly  without  being  able  to  verbalize 
it  entirely.  Because  this  shapinq  process 
is  made  up  of  many  individual  components 
anything  which  would  permit  the  latter  to  be 
learned  separately  may  enhance  training.  An 
example  would  be  the  landing  "flare,"  the 


transition  from  the  approach  to  the  touchdown 
and  roll-out.  This  maneuver  cannot  be  ver- 
balized in  such  a fashion  that  the  student 
can  slavishly  follow  a set  of  instructions. 

He  must  carry  out  a coordinated,  smooth 
maneuver  based  on  the  "feel"  of  the  air- 
craft. Flight  crew  training  personnel  can 
benefit  from  recordings  of  performance 
not  only  in  having  a permanent  record  to 
show  the  student  but  also  to  study  for  the 
purpose  of  finding  additional  cues  to  per- 
formance variation,  cues  which  may  point 
to  a problem  early  on,  before  bad  habits  are 
built  up. 

Recommendations.  Now  that  the  visual  systems 
have  been  installed  on  all  four  Boeing  Flight 
Crew  Training  simulators  (707,  727,  737,  747) 
and  have  been  demonstrated  to  be  effective 
for  simulated  flight,  maximizing  their  uti- 
lity requires  answers  to  questions  for  which 
there  are  no  ready  answers.  For  example, 
how  complete  can  simulator  training  be  made? 
Is  it  conceivable,  given  the  requisite  FAA 
approval,  that  the  pilot  could  be  certified 
in  the  simulator?  If  not,  how  close  to 
this  can  we  get?  Would  the  training  function 
be  further  enhanced  by  increased  computer 
capacity  for  the  visual  system?  Just  how 
well  does  training  in  the  simulator  transfer 
to  the  aircraft? 

The  Boeing  Company  Flight  Crew  Training 
organization  is  developing  plans  for  a 
research  program  to  provide  the  necessary 
data.  These  involve  the  determination  of 
sensitive  measures  of  pilot  performance 
which  will  provide  quantitative  criteria  for 
the  evaluation  of  training.  The  research 
program,  expected  to  extend  over  a period 
of  several  years,  will  use  the  present  visual 
system  to  establish  baseline  Derformance. 
from  this  baseline  both  greater  and  lesser 
degrees  of  completeness  of  simulation  will 
be  evaluated. 

In  summary,  the  task  of  this  research  effort 
is  to: 

1.  Develop  valid  and  reliable  quantitative 
measures  of  pilot  performance  involving 
reliance  on  vision. 

2.  Establish  "baseline"  performance  with 
the  present  visual  system. 

3.  Determine  the  level  of  display  fidelity 
(e.g.,  resolution)  below  the  present 
system  level  which  would  result  in 
significant  degradation  of  pilot  per- 
formance. 

4.  Determine  the  level  of  display  fidelity 
which  would  result  in  significant 
improvement  in  pilot  performance. 

5.  On  the  basis  of  this  research  effort. 
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develop  and  document  a set  of  guidelines 
to  assist  our  commercial  customers  and 
other  interested  parties  in  the  selection 
of  visual  simulation  systems  that  will 
meet  mission  goals  and  at  the  same  time 
be  cost-effective. 

The  first  task,  that  of  developing  valid  and 
reliable  measures  of  pilot  performance,  is 
both  critical  and  difficult.  The  obvious 
place  to  begin  is  with  questions  submitted 
to  senior  instructor  pilots  regarding  their 
current  criteria  for  assessing  pilot  perfor- 
mance, both  in  the  airplane  and  in  the  simu- 
lator. The  data  thus  obtained  can  be 
analyzed  to  develop  correlative  physical 
measures  available  from  instrumentation  on 
the  aircraft  and  on  the  simulator.  A sig- 
nificant portion  of  this  development  effort 
must  be  spent  with  senior  pilots  generating 
test  data  to  establish  validity  and  reli- 
ability. The  resulting  measures  can  then 
be  used  in  a series  of  studies  relating 
simulator  training  to  pilot  performance  in 
the  aircraft. 

APPENDIX  A 

INFORMAL  QUESTIONNAIRE 

Instructions : Would  you  please  provide  us 

with  your  comments  about  the  visual  simu- 
lation now  that  you  have  completed  the 
twelve  approaches  to  Grant.  Co.  Airport. 

You  may  write  your  comments  in  the  space 
provided  below  each  question.  Or  if  you 
prefer  you  may  use  the  tape  recorder  and 
identify  the  tape  to  us  by  the  experimental 
code  number.  CGI-1. 

(1)  How  realistic  did  the  visual  simu- 
lation appear?  Did  your  impression  of  it 
change  with  time  on  the  task? 

(2)  Was  there  anything  annoying  about  it? 

(3)  Did  there  appear  to  be  any  discrepancy 
between  visual  feedback  and  motion  feed- 
back? 

(4)  What  portions  of  current  training  or 
recertification  done  in  motion  base  simu- 
lators do  you  think  might  be  improved  by  the 
addition  of  visual  siumlation? 

(5)  What  portions  of  training  or  recerti- 
fication now  requiring  actual  flight  time  do 
you  think  might  be  done  adequately  with  a 
simulator  having  this  visual  display? 

(6)  Was  there  anything  missing  in  the  visual 
display  that  you  think  should  be  there? 

(7)  Do  you  have  experience  flying  into  Moses 
Lake?  Day?  Night? 


(8)  How  recently  had  you  flown  the  737  simu- 
lator before  the  approaches  you  have  just 
completed:' 

(9)  Would  this  kind  of  simulation  have  any 
particular  value  in  the  training  of  pilots 
who  fit  the  category  of  "minimum  time 
pilots?"  In  what  way? 

(10)  Would  you  speculate  for  us  on  the  pos- 
sibility of  a visual  simulation  like  this 
permitting  the  actual  certification  of  air- 
line pilots  without  any  checkout  time  in 
the  airplane  itself? 

(11)  Do  you  think  that  the  visual  display 
is  critically  evaluated  by  this  test? 

(12)  Have  you  had  experience  with  any  other 
visual  simulators?  Please  compare. 

(13)  Were  the  side  windows  helpful  in  these 
straight  in  approaches?  How? 

(14)  To  help  us  make  a composite  profile  of 
all  16  pilots  that  will  participate  in  this 
evaluation  would  you  provide  the  following 
information  on  your  flying  and  instructor 
experience? 

Overall  number  of  flying  hours  you  have 
logged  


Number  of  hours  logged  in  jet  aircraft 


Number  of  hours  in  737  aircraft 


Air  Transport  aircraft  you  are  qualified  in 


747 

737 

727 

707 

DC- 10 

DC-8 

DC- 9 

Falcon 

Lear 

Others 

APPENDIX  B 

TRAINING  TRIALS  AS  A PRELIMINARY 
EXPERIMENT 

Introduction.  The  research  and  operational 
people  were  concerned  about  the  completeness 
of  the  visual  scene  as  an  information  source. 
It  was  hypothesized  that  if  the  visual  scene 
contained  sufficient  information,  very  ex- 
perienced pilots  could  land  on  such  a scene 
the  very  first  time  they  made  an  approach  to 
the  image. 

The  method.  Four  practice  trails  were  given 
each  of  the  16  pilots  before  they  made  their 
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eight  approaches  of  the  main  experiemnt. 

The  four  training  trials  were  ordered  as 
A1 , B1 . and  A^.  The  A conditions  were 
runs  without  the  visual  scene  but  with  all 
instruments  available  to  the  pilot.  The 
B conditions  also  provided  all  instruments 
and  the  visual  scene,  B being  a day  scene 
and  B2  a night  scene.  The  results  of  the 
two  A conditions  were  combined  so  that  the 
practice  effect  was  an  average  of  the  first 
and  fourth  trial'.  The  two  B conditions 
were  averaged  and  represented  the  second 
and  third  trials.  Such  a counterbalancing 
should  minimize  the  practice  effects  due  to 
presentation  order.  All  procedures  were 
the  same  as  in  the  main  experiment  as 
discussed  earlier,  with  the  one  signifi-, 
cant  difference  that  in  the  training  trials 
barometric  and  radio  altimetry  were  avail- 
able under  all  conditions. 

The  ’’instrument  only"  trials  were  designed 
to  give,  in  addition  to  the  pilots  very 
recent  practice  trials,  a basis  for  experi- 
mental comparison.  Normally,  pilots  never 
fly  to  touchdown  on  instruments  without 
seeing  the  runway,  so  this  experimental 
condition  has  no  operational  counterpart. 

It  provides  only  a measure  of  the  pilot’s 
instrument  flight  skill  at  that  moment  of 
practice,  and  a statistical  comparison  with 
the  instrument  + visual  scene  performance 
trials. 

The  same  dependent  measures  were  used  in  this 
preliminary  experiment  that  were  used  in  the 
main  experiment.  Two  major  comparisons 
were  made:  (1)  The  area  (footprint)  at 

touchdown  and  (2)  The  descent  rate  at  touch- 
down. The  "footprint  area"  was  defined  as 
♦ one  standard  deviation  in  x and  y from 
the  mean  touchdown  point  and  these  values 
converted  into  an  area. 

The  Results.  The  visual  scene  reduced  the 
"footprint  area"  from  453,024  square  feet 
to  32,528  square  feet,  a reduction  of  13.9 
times  due  to  the  presence  of  the  visual 
scene.  The  rate  of  descent  at  touchdown 
was  reduced  from  11.07  (+  = 15.74  to  6.4) 

feet  per  second  to  5.04  (+  - 8.11  to  1.97) 

feet  per  second.  This  magnitude  of  change 
of  6.03  feet  per  second  could  have  occurred 
by  chance  only  once  in  a thousand  repli- 
cations of  this  experiment.  The  comparison 
information  is  that  the  design  limit  for 
the  aircraft  gear  (737)  is  10  feet  per 
second  for  maximum  gross  weight  of  this 
aircraft. 

Summary.  Although  a very  limited  experiment 
in  number  of  pilots,  trials,  and  repli- 
cations, these  data  are  intriguing.  To  main- 
tain one’s  perspective,  the  reader  should 
recall  the  20  knot  cross  wind  which  sheared 
twice  close  to  the  ground  (2001  and  50’  in 
altitude)  and  made  these  approaches 


difficult  even  for  experienced  instructor 
pilots. 

Under  these  difficult  conditions  and  on  first 
exposure  to  the  visual  scene  the  eight  737 
pilots  were  only  86’  short  and  29’  to  the 
right  of  the  glide  slope  intersection  on 
touchdown.  This  would  be  an  excellent  per- 
formance without  the  windshear.  The  rate 
of  descent  at  touchdown  compares  very 
favorably  with  the  NASA  data  on  touchdown 
rate  in  their  Ames  simulation  studies  with 
many  more  trials.  The  assumed  average  touch- 
down rate  of  "good  landinqs"  is  about  3 
feet  per  second  in  air  transport  operations . 
This  is  only  2 feet  per  second  better  than 
these  pilots  did  on  their  first  and  second 
approach  to  the  computer  generated  image. 
These  data  imply  that  the  information  con- 
tent of  this  combination  of  niqht  and  day 
scene  was  adequate  for  experienced  pilots 
to  make  successful  first  trial  landings. 

This  CGI  scene  was  a replication  of  a very 
familiar  runway  and  airport  for  all  these 
instructor  pilots. 

APPENDIX  C 

CGI  PERFORMANCE  EVALUATION 
INSTRUCTIONS  TO  PILOTS 

Purpose  of  Test.  We  are  evaluating  a new 
visual  simulation  system  built  by  General 
Electric  according  to  Boeing  specifications. 
The  visual  scene  is  generated  by  a computer 
program  and  displayed  in  color  on  a cathode 
ray  tube.  This  scene  contains  major  land- 
marks useful  in  navigation.  Perspective 
changes  associated  with  changes  in  aircraft 
attitude  and  position  provide  the  visual 
feedback  to  the  pilot  related  to  his  flight 
path. 

Our  purpose  in  conducting  this  exercise  is 
to  evaluate  this  display  in  as  systematic  a 
fashion  as  conditions  will  permit.  It  is 
not  an  evaluation  of  the  pilot's  flying 
ability  since  this  ability  is  a necessary 
assumption  in  our  analysis  of  capabilities 
of  the  display. 

Procedure.  After  an  initial  practice  period 
to  ensure  familiarity  with  the  task  and 
with  the  appearance  of  the  scene,  you  will 
be  asked  to  make  8 approaches  to  a visual 
simulation  of  Grant  County  Airport,  at 
Moses  Lake.  You  may  recognize  familiar 
landmarks. 

Visibility,  time  of  day,  and  availability 
of  instruments  are  the  major  variables  in 
the  test.  In  each  case  you  will  start  at 
4.7  nautical  miles  out  at  an  altitude  of 
2500  ft.  (MSI).  There  is  a 20  knot  wind 
from  45  degrees  right  of  runway  headinq  at 
starting  altitude.  On  your  approaches  we  are 


247 


asking  that  you  fly  the  simulator  on  the 
basis  of  the  information  directly  avail- 
able to  you.  The  man  in  the  right  seat 
will  not  make  the  standard  callouts;  he 
will  limit  his  assistance  to  configur- 
ation changes  as  you  direct. 

We  are  asking  that  you  select  the  same 
touchdown  point  in  all  conditions,  that 
correspond  to  an  IFR  landing.  Also,  we 
ask  that  you  attempt  to  keep  lined  up 
with  the  runway  centerline  when  the  visual 
scene  is  present.  The  criterion  variable 
will  be  deviation  from  prescribed  flight 
path. 

Mental  Set.  Each  participant  should  view 
the  test  of  this  visual  simulation  as 
though  he  were  a researcher.  This  means: 

1.  Objectivity  (not  prejudiced  for  or 
against  the  display),  2.  Belief  in  the 
goals  of  the  study,  3.  Appreciation  of 
the  value  of  quantitative  analyses,  and 
4.  Confidence  in  the  validity  of  the 
generalizations  to  be  made  from  the  data. 

To  re-emphasize  something  mentioned  earlier, 
this  is  not  a test  of  your  flying  ability 
and  the  data  you  provide  will  not  be  re- 
ported in  any  way  that  reflects  your 
performance  on  this  task  as  an  individual. 

We  respect  the  fact  that  your  professionalisn 
will  yield  the  best  data  obtainable;  that's 
why  you  were  chosen  to  participate. 

Your  assistance  in  this  research  effort 
is  sincerely  appreciated  and  it  is  hoped 
that  the  experience  is  both  interesting  and 
enjoyable. 

APPENDIX  D 

INFORMAL  COMMENTS  BY  EIGHT  PILOTS 
FLYING  THE  BOEING  VISUAL  SIMULATOR 

Relevant  Experience 

• From  8 to  15  thousand  hours  of  flying  (Ave 
10,700  hrs). 


• From  2 to  7 thousand  jet  hours  (Ave.  4,700 

hours). 

• From  150  to  2,300  hours  in  737  (Ave.  960 

hours) . 

All  had  made  a number  of  approaches  to  Grant 
County  Airport,  the  scene  used  in  the  test. 

Real  ism  of  visual  simulation? 

• All  pilots  thought  the  display  was  realistic 
and  good  enough  to  fly  to  without  instru- 
ments. 


What  is  missing? 

• One  pilot  said  that  greater  definition 

would  be  "nice." 

• One  pilot  said  that  the  red  light  was 

missing  from  the  glide  slope  shack. 

• Six  pilots  had  nothing  to  mention. 


Synchronism  be tween  mot i on 

s i mulation  and 

visual  simulation? 

• Three  pilots  mentioned  experience  with 
other  visual  simulations.  These  included 
Vital  II,  Link  Vamp,  Redifon,  and  Novo- 
view.  All  three  preferred  the  Boeing 
visual  simulation  built  by  General 
Electric. 


Distractions  and  "petty  annoyances" 

• The  few  negative  comments  volunteered  by 
pilots  who  served  in  the  test  relate  to 
appearance  of  scintillation  in  parts  of 
picture  at  various  times  or  a tendency 
for  "threshold  size"  objects  to  pop  into 
view.  In  a couple  of  instances  pilots 
noticed  some  streaking.  One  said  it 
reminded  him  of  "tracer  bullets." 
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AUTOMATED  SCORING  OF  INSTRUMENT  FLIGHT  CHECKS 

H.  KINGSLEY  POVENMIRE  and  LCDR  KENT  M.  BALLANTYNE 
U.  S.  Coast  Guard,  Aviation  Training  Center 


U.  S.  Coast  Guard  helicopter  pilots 
receive  annual  instrument  and  emergency 
training  in  the  Variable  Cockpit  Training 
System  simulator.  Many  portions  of  initial 
aircraft  transition  training  are  also  con- 
ducted in  the  simulator.  Initial  instru- 
ment ratings  as  well  as  annual  instrument 
renewals  are  given  on  the  basis  of  simula- 
tor checkrides , which  are  automatically 
scored  by  the  computer.  Scores  are  kept 
for  a3  many  as  12  parameters  at  a time. 

This  paper  discusses  the  first  three  years 
of  experience  with  automated  scoring.  Mod- 
erate correlations  were  found  between  sub- 
jective instructor  scores  and  automated 
scores.  Although  not  useful  for  individual 
pass- fail  decisions,  normative  comparisons 
of  the  automated  scores  are  responsive  to 
changes  in  the  training  program. 

BACKGROUND 

Coast  Guard  helicopter  pi  let  training 
has  been  conducted  at  the  Aviation  Training 
Center  in  Mobile,  Alabama,  since  1967.  A 
study  was  conducted  in  1969  to  determine 
future  Coast  Guard  aviator  training  require- 
ments (Hal)  et  al , 1969).  After  a thorough 
investigation  of  operational  pilot  tasks, 
several  training  goals  were  identified  which 
had  general  applicability  and  might  best  be 
taught  in  a central  location.  Each  training 
goal  was  evaluated  with  respect  to  simula- 
tion technology  available.  Decisions  were 
then  made  concerning  which  simulator  capa- 
bilities might  be  implemented  to  enhance 
training  of  helicopter  pilots  (Caro  et  al, 
1969).  For  example,  very  little  saving  of 
flight  time  was  envisioned  through  use  of  a 
visual  system  because  of  its  limited  ability 
to  simulate  the  water  environment.  On  the 
other  hand,  many  helicopter  malfunctions 
would  require  all  six  degrees  of  freedom  in 
motion  simulation. 

Based  on  these  two  papers,  the  entire 
training  program  was  revised.  Concurrently, 
a flight  simulator  was  designed  and  built. 
Both  efforts  came  together  in  1973  as  the 
Variable  Cockpit  Training  System  (VCTS) 
became  operational.  The  VCTS  utilizes  a 
highly  sophisticated,  computer-based  flight 
simulator  built  by  Reflectone,  Inc.,  which 
consists  of  two  separate  cockpits.  Each 
cockpit  duplicates  one  of  the  two  different 
types  of  Coast  Guard  helicopters. 


SYSTEMS  APPROACH  TO  TRAINING 

Implementation  of  the  VCTS  was  a 
major  shift  away  from  the  traditional  concept 
of  aviator  training  towards  a program  based 
on  the  systems  approach  to  training.  The 
major  concept  underlying  this  new  training 
system  has  been  in  the  area  of  aircraft 
systems  knowledge.  The  present  methods 
stress  teaching  the  student  how  to  operate 
the  system  as  opposed  to  giving  him  detailed 
explanations  about  how  the  system  operates. 

INSTRUCTOR  AS  TRAINING  MANAGER 

The  key  to  success  in  using  this  new 
approach  has  been  the  instructor's  back- 
ground and  the  role  he  plays.  Instructors 
are  carefully  selected  from  a group  of 
operational  pilots  who  have  expressed  a 
desire  to  come  to  the  training  center.  In 
addition  to  extensive  operational  experi- 
ence, each  instructor  pilot  receives  in- 
struction in  managing  the  training  progress 
of  his  students.  While  working  with  a 
group  of  students,  the  instructor  becomes 
involved  in  all  facets  of  the  course  of 
instruction  from  briefing,  through  the  simu- 
lator, to  the  aircraft. 

PEER  TRAINING 

One  instructor  and  two  students  work 
together  as  a team.  Each  student  spends 
half  the  time  in  the  copilot  seat  and  the 
other  half  in  the  pilot  seat  during  simula- 
tor training  periods.  Simulator  flights 
then  become  primarily  practice  and  valida- 
tion sessions  where  students  must  react  as 
a team  to  situations  covered  'n  the  brief- 
ing and  reading  assignments. 

For  those  courses  involving  actual 
flights,  an  additional  instructor  pilot  is 
also  provided.  He  flies  in  another  aircraft 
with  the  second  student.  This  team  stays 
together  until  the  pilots  have  reached 
course  standards  in  the  basic  helicopter 
maneuvers,  instrument  and  emergency  proce- 
dures and  systems  knowledge.  The  students 
then  fly  with  other  instructors  for  training 
in  operational  maneuvers  and  for  aircraft 
and  simulator  check  flights. 

Briefing  and  debriefing  sessions  are 
conducted  in  a dialectic  manner.  Questions 
are  used  to  lead  students  from  one  concept 


251 


to  another.  In  many  cases  one  student  may 
be  able  to  shed  some  light  on  a point  of  con- 
fusion. Since  the  program  makes  use  of  the 
proficiency  based  learning  concept,  the 
length  of  any  given  course  will  vary  con- 
siderably depending  on  the  student's  past 
experiences  and  his  ability  to  progress 
through  the  new  material. 

The  methods  now  in  use  have  resulted 
in  overwhelming  acceptance  by  the  students. 
They  invariably  praise  the  concept  of  per- 
sonalized self-paced  learning. 

TRAINING  COURSES 

Although  the  facilities  and  methods 
described  have  been  used  for  a variety  of 
purposes,  the  major  emphasis  is  on  recur- 
rent training.  Transition  training  and 
initial  qualification  training  are  also 
conducted  on  a regularly  scheduled  basis. 
Students  in  all  courses  are  qualified 
pilots. 

Pilots  who  are  current  in  one  or  both 
of  the  helicoptef  types  come  to  Mobile  an- 
nually for  recurrent  training.  This  in- 
volves one  week  of  intensive  training  in 
instrument  and  emergency  procedures  conduc- 
ted completely  in  the  simulator.  Upon  com- 
pletion, the  pilot's  instrument  rating  is 
renewed  for  another  year.  Approximately 
500  pilots  a year  go  through  this  course  in 
one  cockpit  or  the  other. 

Transition  training  is  offered  in  each 
specific  type  of  helicopter.  Pilots  who 
have  never  flown  the  helicopter  to  which 
they  are  being  assigned  take  from  three  to 
six  weeks  to  fully  qualify,  depending  large- 
ly on  their  rate  of  learning  and  previous 
experience.  Approximately  30  pilots  go 
through  this  course  in  each  helicopter  an- 
nually. 

In  the  third  major  course,  fixed-wing 
pilots  are  given  their  initial  helicopter 
qualification  in  the  HH-52A  single  engine 
helicopter.  Approximately  18  pilots  per 
year  go  through  this  course.  The  end-of- 
course  objectives  for  the  Qualification 
Course  are  the  same  as  for  the  HH-52A 
Transition  Course.  For  planning  purposes, 
an  additional  week  is  allowed  to  acquire 
the  basic  helicopter  skills. 

The  switch  in  training  philosophies 
and  the  implementation  of  the  VCTS  have 
resulted  in  a much  higher  quality  of  train- 
ing at  a reduced  cost.  Emergency  procedures 
are  practiced  in  the  simulator  that  are  not 
possible  in  the  aircraft.  The  simulator 
operating  costs  are  relatively  stable  at  $65 


per  hour  while  aircraft  operating  costs  are 
high  and  rising  rapidly.  Cost  figures  for 
Coast  Guard  helicopters  are  roughly  ten 
times  as  much  for  the  HH-52A  and  15  times  as 
mucn  for  the  HH-3F. 

OBJECTIVE  SCORING  TECHNIQUE 

Recent  interest  in  objective  scoring 
of  pilot  performance  has  grown  out  of  some 
longstanding  uncertainties.  In  the  one-on- 
one  evaluation  situation  imposed  by  most 
training  aircraft  it  is  impossible  to  achieve 
totally  objective  performance  scores.  Train- 
ing supervisors  have  had  to  make  judgments 
concerning  quality  of  training  and  student 
progress  based  on  scores  collected  by  a num- 
ber of  instructor  pilots. 

Many  efforts  have  been  made  through  the 
years  to  increase  the  objectivity  of  scoring 
by  check  pilots  (Smith  et  al,  1952;  Povenmire 
et  al,  1973;  Koonce,  1974).  In  every  case  it 
was  found  that  reliability  between  observers 
increased  when  scores  were  based  on  instru- 
ment readings  rather  than  outside  references 
or  quality  judgments.  With  present  computer 
technology.it  is  quite  simple  to  automati- 
cally score  instrument  readings  much  more 
accurately  than  can  be  done  even  with  an 
expert  human  observer. 

The  Coast  Guard  has  been  collecting 
data  on  a computer  scored  operational  instru- 
ment checkride  for  over  two  years.  We  will 
herein  report  some  preliminary  results  and 
point  out  some  pitfalls  to  others  presently 
designing  similar  systems. 

Automated  Checkride 

The  automated  checkride  incorporated 
in  the  VCTS  simulator  records  frequency  and 
time  out  of  tolerance  for  12  flight  para- 
meters. A simple  computer  language  called 
SANSKRIT  is  supplied  by  Reflectone,  Inc., 
to  allow  Training  Division  personnel  to 
design  checkrides. 

Each  maneuver  is  broken  down  into  seg- 
ments for  scoring  purposes.  Errors  are  ac- 
cumulated and  stored  for  as  many  as  nine 
segments  of  a maneuver.  Each  segment  may  be 
further  divided  into  as  many  as  15  blocks  to 
ensure  proper  sequencing  through  the  segment. 
The  checkride  is  automatically  sequenced  when 
the  student  reaches  the  "END  CRITERIA"  of 
each  block. 

Maneuvers  are  ordered  in  such  a way  as 
to  simulate  an  operational  instrument  mission. 
The  instructor  acts  as  an  FAA  controller  giv- 
ing clearances  and  weather  information  which 
would  be  heard  on  an  actual  flight.  He  also 


records  information  which  will  not  show  up  on 
the  recorded  checkride  such  as  cockpit  proce- 
dures, planning  and  crew  coordination.  Prior 
to  the  checkflight,  the  student  plans  the 
flight  as  he  would  in  the  real  world  includ- 
ing all  required  paperwork. 

A typical  checkride  commences  with  an 
instrument  takeoff  from  Bates  Field  in 
Mobile  destined  for  Gulfport  30  miles  away. 

The  student  in  the  pilot  seat  is  automati- 
cally graded  on  his  takeoff,  his  airways 
tracking  as  he  flies  to  Gulfport,  and  his 
holding  procedures  simulating  delays  for 
traffic  or  weather.  He  will  make  several 
approaches  down  to  Minimum  Descent  Altitude. 

He  is  assisted  by  his  flying  partner  acting 
as  copilot.  Malfunctions  are  automatically 
inserted  along  the  way  to  allow  the  instruc- 
tor to  subjectively  grade  both  crewmembers 
in  their  teamwork,  judgment  and  trouble- 
shooting strategy.  From  an  experimental 
standpoint,  the  copilot  provides  an  unwanted 
source  of  variance.  However ; greater  realism 
and  training  value  are  achieved  by  having 
two  students  act  as  a crew. 

Establishing  Criteria 

Special  attention  must  be  paid  to 
designing  "end  criteria"  for  each  checkride 
block  in  order  to  assure  proper  sequencing. 

End  criteria  must  be  met  not  only  by  any 
acceptable  procedures,  but  by  any  conceivable 
procedure.  If  poor  performances  do  not  p~o- 
duce  a completed  record,  the  lower  end  of  the 
distribution  will  be  eliminated. 

"Scoring  criteria"  must  be  developed 
to  allow  any  of  the  acceptable  alternatives 
to  be  scored  equally.  Many  operational 
instrument  maneuvers,  such  as  procedure  turns, 
allow  alternative  procedures  which  are 
equally  correct.  This  limits  the  number  of 
items  that  can  be  scored  during  a given 
block. 

The  computer  automatically  prints  out 
a record  of  error  scores  for  each  maneuver 
at  the  end  of  the  checkride  provided  that 
all  end  criteria  have  been  met.  These 
records  are  retained  in  permanent  storage 
for  complex  data  analysis  and  reporting. 

ANALYSIS  AND  DEVELOPMENT 

Several  questions  must  be  answered 
before  this  data  can  be  used.  First,  how 
many  records  must  be  collected  before  the 
data  reaches  an  acceptable  level  of  reli- 
ability? Second,  do  error  scores  discrimi- 
nate between  differences  in  pilot  ability? 
Third,  are  these  scores  sensitive  to 
changes  in  the  training  program? 


Reliability 

Each  quarter  we  would  analyze  the  data 
from  all  checkrides  stored  to  date  to  eval- 
uate the  stability  of  the  mean  error  scores. 
Oddly,  these  means  kept  going  down  rather 
than  merely  being  unstable.  We  then  ran 
separate  analyses  grouping  students  chrono- 
logically, each  group  of  20  students  repre- 
senting roughly  eight  to  ten  weeks.  No  stu- 
dent was  scored  twice  during  this  period. 
There  was  a high  correlation  between  time 
and  group  mean  scores  (r  = -.98).  The 
results  of  an  analysis  of  variance  indicate 
a high  reliability  of  this  effect  as  shown 
in  Table  1.  This  phenomenon  has  held  true 
in  five  of  the  six  checkrides  currently 
developed. 

This  can  be  attributed  to  two  factors. 
Pilots  completing  the  training  would  pre- 
sumably discuss  the  sequence  of  events  back 
at  their  home  unit  to  the  benefit  of  those 
pilots  yet  to  come.  This  was  partially 
counteracted  by  the  freedom  of  instructors 
to  substitute  malfunctions  of  the  same  gen- 
eral type  for  those  pre-programmed.  Sec- 
ondly, as  instructors  became  efficient  in 
the  role  of  training  manager  and  more 
familiar  with  the  checkride  sequence,  they 
became  better  able  to  train  students  in  the 
skills  measured  by  the  automated  scoring 
system. 


TABLE  1.  ANALYSIS  OF  VARIANCE  OF  MEAN 
TOTAL  ERROR  SCORES  FOR  CHRONO- 
LOGICAL GROUPINGS  OF  STUDENTS 
ON  HH-52A  CHECKRIDE  1. 


N 

X 

SD 

Feb  74-May  74 

24 

1698.2 

711.2 

Jun  74-Aug  74 

21 

1449.9 

629.2 

Sep  74-Jan  75 

22 

1141 .3 

569  9 

Feb  74-Apr  75 

20 

1053.8 

358.3 

All 

87 

1350.1 

638.6 

df 

'H 

F=5.51 

p<.01 

Validity 

A comparison 

was  made  between 

a group 

of  nine  instructor 

pilots 

, who  are 

res  pons  i- 

ble  for  determining  standard  operating  pro- 
cedures, and  a group  of  23  operational 
pilots  on  their  annual  instrument  checkride. 
This  comparison  showed  significant  differ- 
ences favoring  the  instructors  in  total 
error  score  (p<.00 1)  and  in  38  of  the  58 
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individual  parameter  scores  (p<.01).  Of  the 
twenty  remaining  parameters  scores  where  no 
significant  differences  were  found,  all  but 
two  favored  the  instructors  (Povenmire, 

1974). 

The  objective  scores  did  seem  to  dif- 
ferentiate between  two  groups  known  to  have 
different  levels  of  ability  but  the  question 
remains  as  to  whether  higher  error  scores 
indicate  progressive  poorer  overall  per- 
formance. There  can  be  no  doubt  that  the 
simulator  computer  can  measure  what  it 
measures  with  unfailing  accuracy.  However, 
scoring  parameters  were  restricted  to  those 
that  a well -qualified  instructor  would  con- 
sider critical  if  violated.  As  previously 
mentioned,  the  number  of  scoring  criteria 
were  further  restricted  to  allow  all  cor- 
rect procedures  to  be  scored  equally.  These 
two  factors  severely  limited  the  number  of 
data  points  that  could  be  scored. 

This  brings  up  an  important  concept. 
Totally  objective  evaluation  is  possible 
only  when  limited  to  those  performance 
criteria  that  can  be  expressed  in  quantita- 
tive terms.  Much  to  the  dismay  of  those 
who  write  behavioral  objectives,  judgmental 
behaviors  cannot  be  quantified.  Such  major 
areas  as  troubleshooting  strategy,  reorder- 
ing of  priorities  and  coordination  of  flight 
crew  can  only  be  evaluated  subjectively. 

It  is  generally  assumed  that  some  cor- 
relation exists  between  ability  to  accurately 
perform  standardized  maneuvers  and  judgmental 
maturity  and  sophistication.  To  test  this 
assumption,  we  asked  the  instructors  to  give 
a single  subjective  grade  using  a four  point 
scale  for  the  entire  checkride.  This  was 
done  prior  to  looking  at  the  computer  print- 
out of  error  scores. 

The  correlation  between  total  error 
scores  and  instructor  grades  on  each  current 
checkride  is  shown  in  Table  2.  A perfect 
correlation  would  be  -1.0  due  to  the  fact 
that  higher  subjective  grades  indicated 
better  performance  and  higher  computer  gen- 
erated error  scores  indicated  poorer. 


TABLE  2. 

CORRELATION 

BETWEEN 

TOTAL  ERROR 

SCORE  AND 

SUBJECTIVE  GRADES.  > 

Checkride 

r 

HH-3F 

3 

-.46 

4 

-.56 

HH-52A 

3 

-.63 

4 

-.70 

1 1 

These  correlations  are  somewhat  lower 
than  might  be  expected  between  two  instruc- 
tors subjectively  grading  the  same  perfor- 
mance (Povenmire,  1970;  Koonce,  1974).  They 
are  also  lower  than  those  found  by  Knoop 
(1973)  using  a complex  computed  function  of 
instructor  pilot  performance.  Higher  cor- 
relations might  be  achieved  with  future 
refinement  discussed  below. 

Distribution  of  checkride  scores  in 
each  grade  category  shown  in  Figure  1 illus- 
trates the  wide  areas  of  overlap  between 
various  categories. 


Excellent 


Above 

average 


Below 

average 


Figure  1.  Distribution  of  checkride  error 
scores  in  each  subjective  grade 
category. 


Clearly,  automated  checkride  scores 
cannot  be  the  sole  basis  for  passing  or 
failing  the  final  checkride.  They  can  be 
used  however  to  assist  instructors  in  making 
ass-fail  judgments.  The  greatest  benefit 
of  these  objective  scores  involves  their  use 
in  evaluating  long-term  stability  of  the 
training  program. 


Sensitivity 


Beginning  in  January  1976,  the  instru- 
ment renewal  course  was  changed  by  giving 
the  automated  checkride  on  the  first  flight 
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with  no  warm-up  instead  of  at  the  end  of  the 
course.  If  performance  was  satisfactory, 
the  emphasis  shifted  to  emergency  training. 
If  the  initial  check  showed  weaknesses  in 
instrument  flying  the  instructor-manager 
continued  to  emphasize  IFR  procedures  until 
the  student  performance  met  criteria.  In 
either  case,  student  performance  on  instru- 
ments and  certain  mandatory  malfunctions  met 
end-of-course  criteria  as  before. 

This  change  in  the  training  seguence 
was  immediately  reflected  in  the  mean  error 
scores  for  Doth  checkrides  in  both  cockpits 
as  shown  in  Figure  2. 

DISCUSSION 

Although  limited  in  scope  to  mechani- 
cal piloting  ability  this  automated  perfor- 
mance scoring  system  was  responsive  to  this 
major  change  in  the  training  system.  After 
each  such  change,  however,  a period  of  time 
is  required  to  collect  enough  data  to  eval- 
uate the  effect.  Data  shown  in  the  1976 
portion  of  Figure  2 were  collected  over  an 
18  week  period. 

Giving  the  recorded  checkride  first 
has  given  us  a better  indication  of  general 
operational  readiness  of  Coast  Guard  pilots. 
It  has  also  given  the  instructor-manager 


more  information  on  student  capabilities  and 
therefore  more  flexibility  in  the  conduct  of 
training.  Students  and  instructors  alike 
have  expressed  nearly  unanimous  approval  for 
this  sequence. 

On  the  other  hand  we  have  eliminated 
the  ability  to  judge  the  effectiveness  of 
the  training  program  itself.  Course  re- 
quirements will  not  permit  giving  two  fully 
automated  checkrides  during  this  one-week 
course.  We  are  currently  developing  a very 
short  checkride  to  be  given  at  the  end  of 
the  week,  consisting  of  three  maneuvers  in 
which  aircraft  control  is  weighted  very 
heavily  as  opposed  to  judgmental  behaviors. 

Refinements 

Several  areas  for  further  development 
may  provide  more  accurate  descriptions  of 
total  performance.  There  is  presently  no 
differential  weighting  applied  to  the  fre- 
quency and  time  out  of  tolerance  on  various 
parameters  except  by  reducing  tolerances  at 
critical  points.  A student  receives  the 
same  number  of  error  seconds  if  he  exceeds 
his  cruising  altitude  by  100  feet  as  he  does 
by  going  200  feet  below  minimums  on  an  ILS 
approach.  Conversion  to  modified  standard 
scores  using  standard  deviations  from  an 
optimum  (Koonce,  1974)  seems  promising  as  it 


Figure  2.  Mean  scores  for  each  ten  classes  in  sequence  for  each  current  checkride. 
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tends  to  weight  each  parameter  according  to 
how  much  emphasis  is  placed  on  it  by  the 
sample  of  operational  pilots.  For  this  pur- 
pose, error  scores  would  be  more  meaningful 
in  terms  of  maximum  deviation  and  root  square 
error  rather  than  time  and  frequency  out  of 
tolerance. 

Greater  reliability  might  be  achieved 
in  a more  controlled  flight  profile  than  is 
possible  here.  In  certain  phases  of  pilot 
training  judgmental  behavior  heed  not  be 
evaluated.  End-of-phase  objectives  can  be 
fully  described  by  performance  on  a series 
of  standardized  maneuvers  where  aircraft 
control  parameters  are  most  important. 

CONCLUSIONS 

Although  we  have  relinquished  more  pre- 
cise control  of  the  test  situation  in  favor 
of  more  realism  in  the  test  environment, 
results  show  that  automated  objective  scoring 
is  possitle.  As  more  data  are  collected  add- 
ing increased  stability, we  should  be  able  to 
evaluate  instrument  proficiency  of  Coast 
Guard  aviators  on  a long-term  basis.  With 
the  addition  of  a five  minute  checkride  at 
the  end  of  the  course, we  will  regain  an 
objective  means  for  monitoring  the  training 
program  itself. 
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INTRODUCTION 
The  Evaluation  Problem 


Student  pilots  must  learn  to  operate  a 
complex  system  in  an  unstable,  frequently 
dangerous  environment.  These  operational  fac- 
tors make  stringent  demands  on  an  instructor 
pilot  (IP)  as  he  evaluates  student  pilot  (SP) 
performance  in  an  aircraft.  The  IP  must  mon- 
itor SP  behaviors,  unsafe  performance  condi- 
tions, and  dangers  in  the  airspace.  Observa- 
tions in  all  relevant  areas  of  performance 
must  be  remembered  or  recorded  and  then  used 
to  arrive  at  a meaningful  evaluation  of  per- 
formance skill. 

Clearly,  the  IP  must  process  large 
quantities  of  information.  As  training  tasks 
become  more  complex  or  dangerous,  demands  on 
the  IP  tend  to  increase  and  IPs  tend  to  be 
less  and  less  able  to  adequately  process  all 
the  essential  information.  As  the  informa- 
tion processing  load  increases,  the  integrity 
of  safety  procedures,  training  effectiveness, 
and  evaluation  methods  may  be  compromised. 

Evaluation  Methods  in  Pilot  Training 

After  more  than  30  years,  rating  scales 
remain  the  basic  IP  evaluation  method.  Rat- 
ing scales  are  still  used  because  they  best 
meet  the  needs  of  training  management  and  the 
operational  user  - the  IP  - without  intruding 
seriously  into  the  IP's  operational  tasks 
(Koonce,  1974). 

As  currently  used,  however,  rating  scale 
methods  are  marginally  adequate  for  evalua- 
tions in  training  research  and  development 
because  most  observations  using  them  lack  suf- 
ficient discrimination;  i.e.,  the  observations 
tend  to  accumulate  on  two,  or  at  best  three 
points  in  the  rating  scale  (Knoop  & Welde, 
1973).  To  make  IP  observations  more  discrim- 
inating in  training  research  and  development, 
extensive  observer  training  and  quality  con- 
trol programs  must  be  developed  and  carried 
out  (Horner,  Radinsky,  and  Fitzpatrick,  1970; 
Koonce,  1974).  This  is  a continuing  and 
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cumbersome  process  during  the  design  and  con- 
duct of  a study.  Furthermore,  no  practical 
alternative  to  IP  ratings  of  airborne  per- 
formance has  been  readily  available'.- 

The  increasing  sophistication  of  flight 
simulators  allows  greater  opportunity  to  uti- 
lize objective  data  recording  and  processing 
methods.  When  such  simulators  are  used  in 
both  training  and  research,  it  is  Important 
that  the  objective  measures  be  reliable, 
readily  accessible,  simply  computed  and,  most 
importantly,  meaningful  to  each  user  - IP, 
student  pilot,  and  researcher. 

There  are  a variety  of  objective  meas- 
ures that  can  be  used  with  a flight  simulator. 
In  this  study,  objective  measures  were  divided 
into  two  broad  categories:  exact  and  summary 

indicators.  An  exact  indicator  was  a measure 
easily  observed  at  one  or  a few  points  in  the 
sequence  of  events  which  comprise  the  profile 
of  a training  task.  Most  exact  indicators 
used  in  training  research  are  not  well  suited 
for  general  applications  because  they  are  not 
usually  derived  from  theoretical  or  general 
training  requirements.  Instead,  they  are 
selected  or  developed  to  reveal  potential 
differences  in  performances  which  will  answer 
some  specialized  research  question.  When  that 
specialized  need  has  been  satisfied,  all  fur- 
ther interests  in  the  measurements  are 
dropped  (Koonce,  1974). 

By  contrast,  summary  indicators  are  very 
popular  In  most  training  research  applications 
Selected  to  reveal  general  trends  or  overall 
differences,  a summary  indicator  was  a global 
measure  computed  conti nuousTy  or  from  a large 
number  of  discrete  observations  made  through- 
out a single  performance.  Summary  indicators 
have  been  criticized  because  they  are  usually 
insensitive  to  the  effects  of  particular 
events  or  skills  In  a performance  (Knoop  and 
Welde,  1973),  and  because  they  are  usually 
based  on  massive  quantities  of  data  with  no 
definitely  specified  relationship  to  the  eval- 
uation problem  (Christensen  and  Mills,  1967; 
Koonce,  1974). 

The  purpose  In  this  study  was  to  investi- 
gate exact  indicators  of  particular  skills  or 
events  that  could  be  used  with  a performance 
state  evaluation  model  to  simplify  objective 


measurement  procedures  in  pilot  performances; 
in  short,  to  reduce  excessive  detail  while 
not  being  uni formatively  general  (Youtz  and 
Ericksen,  1947).  A step  in  the  solution  of 
this  detail  versus  generality  diletnna  would 
be  achieved  if  a general  set  of  exact  indi- 
cators were  derived  from  a theoretical  and/ 
or  training  requirements  analysis,  and  if 
such  measures  were  empirically  shown  to  carry 
sufficient  information  to  evaluate  perfor- 
mances in  a training  research  effort.  The 
research  hypothesis  was  that  a set  of  exact 
indicators  would  successfully  replace  a 
global  sumnary  indicator  typically  used  in 
pilot  training  research. 

APPROACH 

The  Performance  State  Evaluation  Model 

Movements  of  an  aircraft  through  space 
and  time  (i.e.,  the  flight  path)  can  be  de- 
scribed with  values  from  several  variables 
such  as  altitude,  heading,  airspeed,  pitch, 
and  power.  To  accomplish  his  mission,  a 
pilot  must  control  the  movements  of  the  air- 
craft by  using  information  from  some  of  these 
variables  to  make  changes  on  others.  Any 
set  of  values  on  these  variables  becomes  a 
system  "state."  A state,  then,  is  the  com- 
plete set  of  variables  needed  to  describe 
the  entire  performance  of  the  pilot  and  the 
aircraft  at  any  instant  in  time  (Etkin, 

1972).  To  evaluate  a pilot's  performance, 
values  are  observed  in  one  or  more  states 
and  if  they  meet  certain  requirements,  the 
performance  is  assigned  an  appropriate  score 
or  mark  to  indicate  the  level  of  achievement. 

A review  of  measures  used  in  previous 
pilot  training  research  produced  three  pos- 
sibilities which  might  be  effectively  used 
as  exact  indicators.  These  three  measures 
were  the  range,  the  maximum  deviation  from 
the  standard,  and  performance  time.  The 
range  and  maximum  deviation  were  established 
in  World  War  II  as  objective  measures  of 
instrument  flying  skills  (Hagin,  1947). 

Since  that  time,  many  researchers  have  vali- 
dated the  utility  of  these  two  measures. 

In  parallel  research  during  World  War 
II,  performance  time  was  also  studied 
(Miller,  1947).  Performance  time  appeared 
promising  as  a simplifying  metric  Inasmuch 
as  it  was  also  found  to  be  a good  indicator 
in  two  recent  studies.  In  a training  ex- 
periment, Brecke,  Gerlach,  and  Shipley  (1974) 
found  that  differences  in  means  and  vari- 
ances on  performance  time  discriminated 
between  treatment  groups.  In  another  study, 
Knoop  and  Welde  (1973)  obtained  87  different 
objective  measures  from  aircraft  data  as 
well  as  ratings  from  pilots  and  student 
pilots  on  the  same  performances.  Of  the  87 
different  measures,  7 or  8%  were  time 


measures.  When  correlated  with  the  ratings, 

11  or  14%  of  the  77  significant  correlations 
(.05)  involved  the  time  measures. 

Performance  Time  Algorithm 

To  empirically  study  time  and  deviation 
measures,  a situation  was  selected  in  which 
students  in  undergraduate  pilot  training  were 
learning  a complex  instrument  flight  task,  the 
Vertical  S-A,  in  a simulator  (USAF  ATC  Sylla- 
bus, 1975).  Performance  state  criteria  were 
analytically  determined  for  seven  "states"  of 
the  Vertical  S-A  by  Brecke  and  Gerlach  (1972). 
Shipley,  Gerlach,  and  Brecke  (1974)  derived 
flight  path  standards  and  performance  times 
from  these  criteria.  In  the  present  study, 
an  algorithmic,  performance  state  evaluation 
model  was  developed  for  the  maneuver  and  per- 
formance times  and  deviations  from  the  stan- 
dard flight  path  values  were  then  studied  as 
potential  indicators  of  skill  in  three  em- 
pirical investigations. 

Since  it  is  impossible  for  a performance 
to  deviate  significantly  from  standard  times 
and  still  satisfy  all  other  performance  cri- 
teria throughout  an  entire  flight,  performance 
times  can  be  used  with  deviations  from  flight 
path  standards  to  simplify  measurement  and 
evaluation  procedures  for  many  maneuvers.  In 
the  algorithmic  model,  if  total  time  is  not 
within  given  tolerances,  some  state  or  states 
must  contain  errors;  these  errors  will  be  re- 
flected as  deviations  from  the  standard.  The 
state  or  states  which  contain  the  errors  can 
be  quickly  located  by  examining  the  perfor- 
mance time  for  each  state.  For  a state  that 
fails  the  time  test  there  are  three  possible 
outcomes;  (a)  if  the  entry  conditions  to 
that  state  deviate  significantly  from  the 
standard,  at  least  some  source  of  the  error 
will  be  in  the  preceding  state  or  states;  or, 

(b)  if  the  entry  conditions  for  a state  are 
within  the  given  tolerances,  the  sources  of 
the  error  must  be  located  in  that  state;  and, 

(c)  if  the  end  point  deviations  are  oITF  of 
tolerance,  the  following  state  should  also 
contain  information  about  the  "pi lot's  per- 
formance. 

Data  for  the  three  investigations  were 
obtained  from  an  earlier  experimental  study 
by  Brecke  (1975)  in  which  measures  of  time, 
airspeed,  heading,  vertical  rate,  pitch, 
power,  and  altitude  were  automatically  ob- 
tained, at  a sampling  rate  of  one  second, 
from  performances  of  the  Vertical  S-A  ma- 
neuver in  a flight  simulator.  (Readers 
interested  in  more  detail  about  the  experi- 
ment or  the  data  should  consult  AFOSR  Tech- 
nical Reports  #50430  and  #60229  by  Brecke 
(1975)  and  Shipley  (1976)  respectively.) 
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INVESTIGATION  I 

The  first  investigation  was  carried  out 
to  compare  "standard"  time  values  for  the 
Vertical  S-A  states  with  observed  perfor- 
mances from  two  experienced  pilots.  The  pur- 
pose was  to  discover  whether  or  not  the 
performance  state  evaluation  model  would  re- 
veal any  significant  differences  among  the 
performances  of  these  two  pilots.  If  the 
model  detected  significant  differences  among 
performances  of  experienced  pilots,  it  should 
also  make  similar  discriminations  among  per- 
formances of  student  pilots.  Such  discrimi- 
nations are  essential  to  effective  evaluation 
in  regular  training  and  in  experimental  re- 
search on  the  effects  of  training  methods. 

Method 

Two  experienced  pilots  performed  a 
sequence  of  12  trials,  6 each,  on  the  Verti- 
cal S-A  in  a flight  simulator  according  to 
procedures  described  by  Brecke  (1975).  Two 
trials  of  one  pilot's  performances  were  not 
included  in  this  investigation;  his  first 
trial  was  a descending  rather  than  a climb- 
ing Vertical  S-A  and  data  from  his  last  trial 
was  unusable  because  of  a data  recorder  mal- 
function. 

Procedures 

Performance  time  and  maximum  altitude 
data  were  obtained  from  computer  printouts 
of  these  performances  and  the  means  and  var- 
iances were  then  computed  for  each  pilot's 
data  on  total  time,  times  for  the  seven  per- 
formance states,  and  maximum  altitude. 
Leviations  of  observed  from  corresponding 
standard  values  were  also  computed  and  used 
as  the  basis  of  tolerance  limits  in  subse- 
quent analyses  of  student  performances. 

Resul ts 

The  means  and  variances  of  the  perfor- 
mance times  revealed  three  significant 
differences.  There  were  two  significant 
differences  (.05)  between  means;  a differ- 
ence of  14.35  seconds  on  total  time  and  a 
difference  of  10.08  seconds  on  the  time  for 
the  climb  state.  Finally,  the  variances 
were  not  homogeneous  for  time  in  the  third 
transition  state. 

INVESTIGATION  II 

The  second  investigation  was  carried 
out  to  determine  whether  total  performance 
time  and/or  maximum  altitude  discriminated 
between  performances  of  treatment  groups  in 
a training  experiment.  An  £ £riori_  predic- 
tion, that  differences  among  maximum  altitude 
variances  would  discriminate  among  treatment 
group  performances,  was  based  on  a training 
requirements  analysis  (Brecke  and  Gerlach, 


1972;  Gerlach,  Brecke,  Reiser  and  Shipley, 
1972)  and  on  the  outcomes  of  a prior  experi- 
ment (Brecke,  Gerlach  and  Shipley,  1974). 

Method 

Thirty-nine  student  pilots  were  randomly 
assigned  to  one  of  five  groups.  Members  of 
four  experimental  groups  studied  the  object- 
ives and  different  preflight  instructions  on 
how  to  perform  the  Vertical  S-A;  members  of 
a control  group  studied  only  the  objectives. 
After  each  subject  had  studied  his  assigned 
materials,  he  performed  a sequence  of  six 
trials  on  the  Vertical  S-A  according  to  pro- 
cedures described  by  Brecke  (1975). 

Procedures 

Total  performance  time  and  maximum  alti- 
tude values  were  obtained  from  the  Brecke 
data.  Total  performance  time  values  were 
analyzed  with  a two  between-  one  within- 
subjects  analysis  of  variance;  repetition  of 
performance  trials  on  the  Vertical  S-A  ma- 
neuver was  the  within  subjects  variable. 
Dunnet's  method  (Myers,  1966)  was  used  to 
compare  the  means  of  the  treatment  groups 
with  the  means  of  the  control  group. 

It  was  hypothesized  that  subjects  in 
groups  given  experimental  instructions  would 
exhibit  less  variability  among  their  per- 
formances at  maximum  altitude  than  subjects 
in  either  the  control  group  or  the  groups 
given  current  instruction.  To  test  this 
hypothesis,  F-ratio  tests  for  differences 
between  group  variances  were  used  rather  than 
tests  for  differences  in  means,  i.e.,  ANOVA 
or  t-tests  (Winer,  1972). 

Resul ts 

Total  time.  In  the  ANOVA  on  total  time, 
there  were  significant  main  effects  for  type 
of  instruction,  number  of  response  items  in 
the  instructional  material,  and  performance 
trials.  A significant  interaction  was  also 
found  between  number  of  response  items  and 
performance  trials.  There  were  no  signifi- 
cant differences  on  Dunnett's  tests  between 
any  treatment  group  and  the  control  group. 

Maximum  al tl tude.  As  predicted,  the 
comparisons  among  the  treatment  groups'  vari- 
ances were  significant  on  the  maximum  alti- 
tude measure.  At  maximum  altitude,  the 
variances  of  the  two  groups  that  received  ex- 
perimental instructions  were  signif icantly 
smaller  (.05)  than  the  variances  of  the  con- 
trol group  and  the  two  groups  receiving 
current  instructions.  The  control  group 
variance  was  not  significantly  different  at 
maximum  altitude  than  the  variances  of  either 
group  receiving  current  instruction. 
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Summary 

To  summarize  the  results  of  the  first 
two  investigations,  performance  times  and 
maximum  altitude  discriminated  among  the  per- 
formances of  both  individuals  and  groups.  To 
the  extent  that  the  observed  variations  in 
performance  time  and  maximum  altitude  reveal 
variability  in  the  pilots'  pitch  and  power 
control  skills,  these  findings  provide  strong 
support  for  the  hypothesis  that  exact  indi- 
cators could  be  used  as  alternatives  to 
summary  indicators.  In  the  third  and  final 
investigation,  this  hypothesis  was  examined 
by  comparing  the  relationship  between  values 
on  the  two  types  of  indicators. 

INVESTIGATION  III 

In  this  last  investigation,  the  hypo- 
thesis was  that  a set  of  exact  indicator 
values  would  predict  values  of  a sunnary 
indicator.  If  scores  from  a sunnary  indica- 
tor in  the  Brecke  (1975)  data  could  be 
predicted  with  scores  from  a set  of  exact 
indicators,  evidence  would  be  obtained  to 
support  a reconnendation  to  use  the  exact 
indicators  in  place  of  the  summary  indicator. 

A multiple  regression  analysis  was  used  to 
test  this  hypothesis. 

Method 

To  evaluate  the  results  of  his  experi- 
ment, Brecke  (1975)  used  a summary  indicator 
of  variability  in  pilot  performances  called 
error  amplitude.  Error  ar  litude  scores  in 
Brecke's  study  were  compul.d  with  procedures 
described  by  Shipley,  et  al.  (1974).  Con- 
ceptually, error  amplitude  is  a normalized 
root  mean  square  measure  obtained  with  the 
deviations  of  each  observation  from  the  cor- 
responding standard  and  a set  of  tolerance 
limits  for  each  parameter.  In  the  present 
investigation,  error  amplitude  scores  from 
Brecke's  data  were  used  as  the  criterion 
variable. 

The  exact  indicators  used  as  variates 
in  the  regression  analysis  were  deviations 
from  the  standard  on  the  performance  state 
times,  total  time,  and  maximum  altitude. 

Three  combinations  Of  performance  state 
times  were  also  included  as  variates  in  the 
regression  design;  these  combinations  were 
(a)  the  sum  of  the  first  three  performance 
state  times;  (b)  the  sum  of  the  last  three 
performance  state  times;  and  (c)  the  sum  of 
the  times  for  all  seven  performance  states. 
These  sums  were  included  in  the  design  to 
test  whether  gross  measures  of  time  would  be 
better  predictors  than  more  specific  measures. 
Prior  experience  measures  Included  as  vari- 
ates in  the  design  were  total  hours  as  a 
pilot,  flying  time  in  undergraduate  pilot 
training,  total  hours  in  the  T-4  simulator, 
and  number  of  minutes  to  complete  the 


instructional  materials  in  the  Brecke  (1975) 
experiment. 

Procedures 

A set  of  30  correlation  matrices  formed 
the  basis  of  the  regression  analysis:  one 

matrix  for  each  of  six  performance  trials 
across  the  five  groups  in  Brecke's  experi- 
mental design.  The  correlation  coefficients 
were  averaged  over  the  five  groups,  after  a 
Fisher's  r to  Z transform  (Hays,  1963),  and 
the  means  were  then  converted  back  to  equiv- 
alent £ values.  The  result  was  a set  of  six 
matrices  of  average  correlations,  one  matrix 
for  each  trial.  Finally,  each  of  these 
matrices  were  submitted  to  a sequence  of  two 
stepwise  regression  analyses.  The  first 
analysis  was  based  on  the  average  correla- 
tions from  the  exact  indicators  and  the  sums; 
the  second  was  based  on  the  average  corre- 
lations from  the  exact  indicators,  the  sums, 
and  the  prior  experience  measures. 

Results 

In  the  results  of  the  regression  analy- 
ses, support  would  be  obtained  for  the 
hypothesis  if  two  criteria  were  satisfied: 

(a)  if  a small  set  (4  to  6)  of  the  12  exact 
indicators  were  consistently  selected  in  the 
regression  equations;  and  (b)  if  equations 
using  these  consistently  selected  indicators 
would  account  for  a substantial  proportion  of 
the  variance  (50%  or  more)  In  the  criterion. 

The  outcomes  of  the  regression  analyses 
confirmed  the  hypothesis.  It  was  found  that, 
of  16  possible  indicators  and  experience 
measures,  a set  of  9 were  frequently  selected 
in  the  regression  analyses.  Various  combi- 
nations from  this  set  of  nine  were  selected 
in  equations  of  just  five  variables  for  each 
of  the  trials  and  these  five  variable  equa- 
tions accounted  for  62%  or  more  of  the  vari- 
ance in  the  summary  indicator,  error 
ampl i tude. 

The  type  of  variable  selected  in  these 
equations  with  five  variables  changed  across 
the  trials.  For  the  first  two  trials,  prior 
experience  and  combined  times  were  predomi- 
nant in  the  equations.  By  the  last  two 
trials,  the  exact  indicators  were  predominant. 

As  measures  of  learning  over  trials,  the 
suninary  indicator  and  the  exact  Indicators 
were  compared  by  making  a plot  of  the  means 
of  each  variable  across  the  trials.  It  was 
found  that  deviations  from  maximum  altitude 
exhibited  virtually  the  same  learning  trend 
over  trials  as  error  amplitude  (jr  = .98). 

These  results  provided  convincing  evi- 
dence to  support  the  hypothesis  that  in  the 
Vertical  S-A,  exact  indicators  could  be  used 
to  replace  a summary  indicator. 


DISCUSSION 

In  these  analyses,  exact  indicators  were 
found  more  sensitive  to  the  effects  of  dif- 
ferent experimental  treatments  than  a suimiary 
indicator.  In  addition,  when  used  alone,  a 
set  of  exact  indicators  were  found  to  account 
for  moderate  (34”)  to  large  (8250  proportions 
of  the  variance  in  a suimary  indicator. 

These  findings  are  provocative  because  they 
suggest  that  a few  well  chosen  observations, 
i.e.,  fewer  data  points,  can  be  used  to  ob- 
tain evaluations  superior  to  those  obtained 
from  typical  suimary  indicators  based  on 
many  data  points. 

The  results  of  these  investigations  also 
support  the  use  of  a performance  state  eval- 
uation model.  Performance  times  and  devia- 
tions from  maximum  altitude  were  found  to 
discriminate  precisely  between  both  individ- 
ual and  group  performances  and  to  identify 
the  sources  of  errors  within  a given  individ- 
ual performance.  It  is  interesting  that 
performance  time  for  the  transition  from 
climb  to  maximum  altitude  was  consistently 
found  among  the  predominant  predictors  in 
the  regression  analyses.  In  previous  re- 
search, Gerlach,  et  al . (1972)  predicted,  on 
the  basis  of  a theoretical  and  a training 
requirements  analysis,  that  this  location  was 
the  most  likely  source  of  student  performance 
errors  in  the  Vertical  S-A.  Coupled  with  the 
demonstrated  sensitivity  of  deviations  from 
maximum  altitude,  it  is  clear  that  the 
evaluation  model  could  be  effectively  uti- 
lized for  performance  of  the  Vertical  S-A  in 
a flight  simulator. 

For  the  operational  user,  these  find- 
ings suggest  that  exact  indicators  can  be 
used  to  replace  sunmary  indicators  and  also 
global  ratings.  Maximum  altitude,  an 
example  of  an  exact  indicator  in  the  Verti- 
cal S-A,  was  found  to  carry  nearly  all  the 
information  given  by  error  amplitude,  a 
summary  indicator.  Since  maximum  altitude 
in  the  Vertical  S-A  occurs  at  a well  defined 
point,  IPs  could  easily  observe  it  and  use 
its  value  to  replace  a global  rating.  The 
dilemma  of  excessive  detail  versus  uninform- 
ative generality  in  pilot  training  measure- 
ment and  evaluation  can  be  given  a workable 
solution  to  the  extent  that  similar  values 
and  their  locations  can  be  derived  for  other 
pilot  training  maneuvers. 
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INTRODUCTION 

Prepare  maintenance  technicians  to  be 
.job-ready.  That  is  the  goal  of  maintenance 
training.  As  it  exists  today,  it  works  with 
varying  degrees  of  proficiency.  Much  of  it 
could  work  better  if  all  the  necessary  train- 
ing aids  and  devices  were  available  to  the 
training  facility:  Audiovisual  devices. 

Maintenance  Training  Units  (MTU's),  Actual 
Equipment  Trainers  (AET's),  Simulation  Models, 
etc.  How  and  what  devices  are  selected  for  a 
particular  application  is  an  exercise  in  it- 
self, and  as  such  is  often  the  reason  why  many 
training  facilities  don't  have  the  necessary 
training  aids.  Effort  is  spent  evaluating  and 
determining  which  or  what  device  best  suits  a 
particular  need.  In  the  meantime  the  training 
facility  is  making  do  with  whatever  it  has. 

This  is  the  case  with  maintenance  sim- 
ulation. Simulation  is  fairly  well  accepted 
for  operator  training,  especially  in  flight 
training.  It  has  its  obvious  advantages--cost 
savings,  safety,  condensation  of  time,  oppor- 
tunity to  practice  procedures  which  occur  only 
during  rare  and  critical  situations,  etc. 

There  is  much  debate  over  how  many  degrees  of 
motion,  how  to  provide  the  motion,  and  what 
fidelity  should  be  provided.  But  whatever  is 
required  or  desired  is  available  - at  a price. 
Simulation  in  other  areas  has  been  slow  to  be 
accepted,  especially  in  maintenance  training. 
One  reason  is  that  it  is  not  always  applicable 
to  the  training  requirement,  but  in  those  in- 
stances where  it  is  applicable,  too  often  the 
reasons  for  not  using  it  are  not  objective. 

It  is  often  accepted  conceptually,  but  it  is 
something  new,  it  is  different,  and  not  every- 
one who  has  the  requirement  is  willing  to  make 
that  step  to  simulation  for  maintenance  train- 
ing. Many  people  are  studying  the  problem. 

They  ask  how  much  fidelity;  what  is 
the  transfer  rate;  what  is  the  retention  rate; 
how  does  it  compare  to  existing  processes;  is 
it  better  than  using  maintenance  training 
units  or  actual  equipment  trainers;  how  best 
to  evaluate  its  applications?  All  valid  ques- 
tions, but  the  questions  continue  to  outnumber 
the  answers.  Some  of  these  people  appear  to 
be  making  the  study  of  the  problem  an  end  in 
itself. 

An  attempt  to  justify  simulation  in 
maintenance  training  will  not  be  made  here; 
however,  some  studies  which  have  been  conduc- 


ted with  the  particular  device  discussed  in 
this  paper  are  referenced:  [1],  (2),  [3],  [4], 

[ 7 j . The  main  intent  of  this  paper  is  to  de- 
scribe the  use  of  a simulator  in  maintenance 
training  at  a facility  that  had  a need  and 
took  the  necessary  steps  to  acquire  it.  That 
facility  is  the  Marine  Corps  Air  Station, 

Cherry  Point,  North  Carolina,  NAMTD  1006,  where 
AV-8A  Organizational  and  Intermediate  Mainte- 
nance Activity  training  is  conducted. 

BACKGROUND 

The  AV-8A  (Harrier)  Aircraft  is  manu- 
factured by  Hawker-Si ddley  Aircraft,  Ltd.,  in 
the  United  Kingdom.  It  is  a high-performance 
attack  jet  aircraft  with  many  complex  inter- 
nal operating  systems.  The  aircraft  is  oper- 
ational with  the  United  States  Marine  Corps, 
Marine  Air  Group  32(MAG-32). 

In  early  1973,  maintenance  training 
for  the  AV-8A  was  being  conducted  at  the 
Marine  Corps  Station,  Beaufort,  South 
Carolina.  The  programs  of  instruction  for 
most  of  the  courses  were  still  in  a develop- 
mental stage.  Training  was  being  conducted 
with  only  a very  limited  number  of  training 
aids.  These  were  primarily  static  display 
panels  and  a few  film  transparencies.  They 
allowed  the  instructor  to  provide  the  student 
with  chalkboard  talks  and  information  about 
the  various  systems  and  their  operation.  The 
instructor  was  able  to  demonstrate  limited 
systems  operation  and  he  could  only  discuss 
the  theory  of  fault  isolation  procedures.  If 
and  when  he  had  test  equipment  available,  he 
was  able  to  demonstrate  calibration,  checkout, 
and  test  maintenance. 

The  scnool  was  having  the  usual  con- 
test with  squadron  maintenance  trying  to  ac- 
quire test  equipment  for  training  use.  As  is 
typical  the  school  was  usually  on  the  short 
end.  In  those  labs  where  they  did  have  test 
equipment,  they  had  the  also  typical  problem 
of  maintaining  and  calibrating  it.  When 
their  test  equipment  was  sent  to  the  calibra- 
tion lab,  it  received  the  lowest  priority, 
and  as  a result  there  were  long  periods  of 
time  when  the  class  would  be  without  test 
equipment.  When  it  was  available  in  the 
classroom  there  was  instructor  reluctance  to 
allow  the  student  free  use  of  it.  The  stu- 
dent had  to  be  continually  monitored.  His  in- 
experience with  using  the  equipment  generally 
resulted  in  its  rapid  deterioration.  Acquisition 
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of  a maintenance  training  simulator  would 
eliminate  this  problem. 


SIMULATOR  DESCRIPTION 


I 


A number  of  simulation  models  which 
could  be  used  in  the  various  courses  were 
proposed  by  Educational  Computer  Corporation 
(ECC).  The  models  were  designed  to  satisfy 
training  requirements  imposed  by  the  instruc- 
tional staff  and  be  able  to  be  used  within 
the  existing  courses  and  those  that  were  yet 
to  be  developed.  The  use  of  the  simulators 
would  increase  the  overall  effectiveness  of 
the  training  by  allowing  the  instructors  to 
cover  each  system  more  thoroughly.  The  simu- 
lation models  would  be  designed  to  include 
the  necessary  test  equipment  in  each  particu- 
lar application  and  thus  eliminate  the  re- 
quirement for  the  actual  special  support  test 
and  calibration  equipment  required.  The  stu- 
dent could  be  allowed  free  use  of  the  models 
for  practice  without  fear  of  damage. 

In  addition,  the  school  was  under 
pressure  to  begin  its  training.  Expediency 
in  delivery  of  equipment  was  critical.  Sub- 
sequently, nine  simulation  models  which  would 
be  designed  to  support  the  training  require- 
ments in  selected  courses  of  instruction,  were 
proposed  by  Educational  Computer  Corporation. 

A contract  was  signed  in  May  of  1973.  In 
January  1974  the  nine  models  were  delivered 
and  accepted.  The  mainframes  had  been  de- 
livered prior  to  that  time.  In  approximately 
28  weeks  after  contract  the  school  had  its 
simulations. 

Originally,  nine  simulation  models  of 
the  operating  AV-8A  Aircraft  Electrical  Sys- 
tems were  proposed.  These  would  be  used  to 
support  organizational  level  maintenance 
training  at  one  site  which  was  later  trans- 
ferred to  MCAS  Cherry  Point,  North  Carolina. 

The  simulation  models  would  enable  the  in- 
structor and/or  the  student  to  locate,  operate, 
calibrate,  test,  and  simulate  the  replacement 
of  aircraft  line  replaceable  assemblies.  The 
nine  models  proposed  were: 

Electrical  Power  Supply  - AC-4KVA 

Power  Supplies  - DC  - 4KVA 

Power  Supplies  - AC/DC  12KVA 

Gas  Turbine  Starter/Auxiliary  Power 

Unit 

Power  Plant  Instrumentation 
Fuel  System  1 
Fuel  Panel  2 
Auto  Stabilization 
Centralized  Warning  System 

The  use  of  these  simulation  models 
subsequently  led  to  the  ordering  of  eighteen 
additional  simulation  models,  two  of  which, 
the  "Air-Data  Computer"  and  the  "Engine  Life 
Recorder"  were  designed  for  Intermediate  Main- 
tenance Activity  (or  "I"  level)  training. 


At  this  point  it  would  be  well  to  de- 
scribe what  the  simulation  models  are.  The 
models  mentioned  are  components  of  a system 
manufactured  by  Educational  Computer  Corpo- 
ration. Detailed  descriptions  of  the  develop- 
ment and  design  of  a simulation  model  are 
available  in  other  papers:  The  Educational 

Computer  Corporation  Approach  tcTTiands-On 
Maintenance  Training,  Siecko  | 5 ] ; Effective 
Training  Through  STmulation--Now,  Siecko  and 
BresTin  [6].  Figure  1 provides  a description 
of  an  EC  II-LP  and  its  various  subcomponents. 
Briefly,  it  is  a main  frame  which  houses  a 
computer  which  is  used  to  drive  the  display 
panel , the  projector  and  the  meter  as  re- 
quired. The  control  console  provides  a means 
whereby  the  overall  status  of  the  simulation 
display  panel  can  be  changed;  for  example, 
from  a normal  configuration  to  a malfunction- 
ing configuration,  or  requirement  for  various 
checkout  procedures  or  a checkout  of  a system 
which  contains  a malfunctioning  component. 

The  display  panel  is  the  central  focus  of  the 
simulation  model  in  that  it  contains  a fac- 
simile rendering  of  the  system  or  systems 
being  simulated.  This  panel  may  also  contain 
hardware  as  is  required  to  facilitate  the 
simulation.  Figure  2 is  a picture  of  the 
simulator  with  a simulation  model  inserted. 
Figures  3 and  4 depict  the  desk-size  version 
of  the  EC  II. 

What  is  unique  about  these  simulators 
is  that  they  are  programmable  and  the  point 
of  interaction  with  the  operator  (student  or 
instructor)  is  the  two-dimensional  panel 
which  contains  a rendering  of  the  prototype 
system  or  device  being  simulated.  This  panel 
may  contain  lifelike  or  actual  hardware  de- 
pending upon  the  training  application.  This, 
in  turn,  is  supplemented  with  a random  access 
slide  projector  to  provide  cues,  symptoms, 
instruction,  queries,  and  so  forth,  to  the 
student  or  directions  to  the  instructor.  Op- 
eration of  the  simulation  model  is  performed 
directly  upon  the  panel.  For  example,  the 
student  or  instructor  would  operate  or  per- 
form a checkout  procedure  of  a radar  or  nav- 
igational control  system  just  as  he  would  in 
real  life.  Control  of  the  simulation  is 
managed  by  an  internal  digital  processor. 

In  addition  to  the  panel  I/O,  there  is 
a control  panel  which  directs  the  status  or 
condition  of  the  simulation.  This  status  can 
take  many  forms.  One  is  that  of  a normal  con- 
di tion--where  all  operations  reflect  the  sys- 
tem in  a normal  or  operating  state.  Other 
conditions  can  be  various  malfunctioning 
states  of  the  simulated  system.  Under  these 
conditions,  all  indicators,  components,  and 
tests  depicted  or  made  on  the  panel  behave  as 
they  would  in  the  real  world  when  the  system 
has  a specific  malfunction.  The  device  can 
accurately  reflect  selected  equipment  problems 
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and  a student  can  repeatedly  practice  his 
diagnostic  skills  until  he  reaches  some 
required  proficiency  level. 

The  operation  of  the  simulation  can  be 
either  directed  or  random.  This  is  an  option 
of  the  simulation  designer  and  is  dependent 
upon  the  governing  factors  within  the  train- 
ing system.  These  factors  are  uncovered 
during  the  design  of  the  training  system.  Most 
often,  when  troubleshooting  has  been  selected 
as  one  of  the  learning  criteria,  the  student 
has  been  relegated  to  the  study  of  flow- 
charts, OJT,  and  reliance  on  his  ability  to 
apply  his  theoretical  understanding  to  the 
real  world.  This  device  provides  simulated 
practice  in  troubleshooting  and  diagnostics  in 
a controlled  learning  environment. 

The  programmability  of  this  type  of 
simulator  provides  it  with  unlimited  potential. 
For  example,  while  the  panel  remains  unchanged, 
the  control  console  or  a separate  program  can 
request  a condition  which  is  fully  instruc- 
tional-introducing the  student  to  the  simu- 
lated system  and  diagnostic  techniques. 

Another  condition  can  be  such  that  the  student 
is  left  to  his  own  capabilities-without  in- 
structional control.  The  program  can  be  con- 
structed to  monitor  the  student  as  much  as  is 
desired.  Modification  of  the  simulation  model 
is  inherent  within  its  design.  The  program 
can  be  modified  inexpensively  and  at  any  time. 

A modification  of  the  panel  generally  in- 
volves only  a patch  to  change  artwork  and/or 
the  addition  or  removal  of  some  panel  com- 
ponent. 

The  device  is  built  by  Educational 
Computer  Corporation  in  various  forms.  One 
model  is  produced  in  a desk-size  configuration 
(the  ECU).  A second  is  a larger  version 
used  for  classroom  demonstration  (the  EC  II- 

LP). 

APPLICATION  OF  SIMULATION  MODELS 

The  NAMTD  1006  receives  most  of  its 
students  directly  from  the  Naval  Aviation  "A" 
School,  Memphis.  An  entering  student  is  nor- 
mally assigned  to  one  course.  After  comple- 
tion he  then  receives  on-the-job  training 
(OJT)  at  the  squadron  and  only  after  his  Work 
Center  Supervisor  deems  he  is  capable  of  un- 
supervised activity  does  he  receive  his  MOS 
rating.  He  may,  however,  be  sent  back  to  the 
school  for  other  speciality  training  and  thus 
have  more  than  one  MOS. 

Courses  at  the  NAMTD  range  from  four 
weeks  for  the  Aircraft  Engine  Mechanic 
Organizational  Course  to  13  weeks  for  the 
Electrician  Organizational  Course.  The  school 
is  authorized  to  teach  26  courses,  but  only  13 
courses  utilize  the  simulation  models,  of 
which  they  have  27  (See  figures  5,  6,  7,  and 
8 for  pictures  of  the  display  panels  for  some 


of  these  models . ) : 

- Nose  Wheel  Steering 

- Head-Up  Display  System 

- 4KVA  DC  Power  Supply 

- LAU-7/A  Guided  Missile  Launcher 

- Engine  Fuel  System 

- CNI  (Navigation) 

- Landinq  Gear 

- Wheel  Brakes  and  Antiskid 

- No.  1 Hydraulic  Power  Supply 

- Wing  Flaps 

- Cabin  Air  Conditioning 

- Armament  Control  System 

- Gas  Turbine  Starter/Auxiliary  Power  Unit 

- 4KVA  AC  Electrical  System 

- 12KVA  AC/DC  Electrical  System 

- No.  2 Hydraulic  Power  and  Ram  Air  Emer- 
gency Supplies 

- CNI  (Communi cations) 

- Air  Brake 

- No.  1 Fuel  System 

- No.  2 Fuel  System 

- Centralized  Warning  System 

- AV-8A  Autostabilization  System 

- Power  Plant  Instrumentation 

- Baseline  800  Attitude  and  Heading 
Reference  System 

- Baseline  800  Weapon  Aiming  System 

- Air  Data  Computer 

- Engine  Life  Recorder 

Because  the  EC  II-LP  simulator  main 
frame  is  a standard  item  and  all  of  the  sim- 
ulation models  are  interchangeable,  the 
school  requires  only  nine  simulator  mair 
frames  to  support  its  27  different  simulation 
models.  The  main  frames  are  portable  and  are 
moved  to  the  classroom  as  they  are  needed, 
depending  upon  course  scheduling  and  student 
flow.  The  appropriate  model-display  panel, 
program,  and  slide  disk  are  then  inserted 
and  the  simulators  are  ready. 

All  of  the  simulation  models  are  pro- 
grammed for  normal  operation  status  and  con- 
tain malfunctions  selected  to  intensify  the 
training  requirements.  All  courses  which 
utilize  the  simulators  reflect  their  use  in 
their  programs  of  instruction. 

The  following  is  a list  of  courses 
taught  at  the  NAMTD  1006  that  utilize  the 
simulator  and  the  simulation  models  used  in 
each  course. 

Pilot's  AV-8A  FAM:  A seven-day  course 
which  utilizes  approximately  20  of  the 
avai lable  models. 

Enlisted  AV-8A  FAM:  A four-day  course 

which  uses  fourteen  of  the  models. 

"0"  - Level  Main ten ance  AV-8A 

Electrician  Organizational  Course:  A 

six-week  course  using  nine  models 


primarily  designed  for  electrician 
training: 

Gas  Turbine  Starter/ Auxi  1 iary  Power  Unit 

4K.VA  DC  Power  Supply 

4KVA  AC  Electrical  System 

12KVA  AC/D C Electrical  System 

Centralized  Warning  System 

No.  1 Fuel  System 

No.  2 Fuel  System 

Power  Plant  Instrumentation 

AV-8A  Autostabilization  System 

Five  hydraulic/electrical  models: 

Wing  Flaps 
Landing  Gear 

Wheel  Brakes  and  Antiskid 
Air  Brake 

Nose  Wheel  Steering 

One  Environmental/Electrical: 

Cabin  Air  Conditioning 

One  INAS/Electrical 

Baseline  800  Attitude  and  Heading 
Reference  System 

Armament  Organizational  Course  - utilizes 
two  panels: 

Armament  Control  System 
LAU-7/A  Guided  Missile  Launcher 

Environmental  Organizational  Course  - 
uti 1 izes  one  panel : 

Cabin  Air  Conditioning 

INAS  (Initial  Navigation  Attack  System) 
Organizational  Course  utilizes  3 panels 

Baseline  800  Weapon  Aiming  System 
Baseline  800  Attitude  and  Heading 
Reference  System 
Head -Up  Display  System 

Hydraulics  Organizational  Course  - 
utilizes  seven  panels: 

No.  2 Hydraulic  Power  and  Ram  Air 
Emergency  Supplies 
Wing  Flaps 
Landing  Gear 

Wheel  Brakes  and  Antiskid 
Air  Brake 

Nose  Wheel  Steering 

No.  1 Hydraulic  Power  Supply 

CNI  (Communication,  Navigation  and 
Identification)  Organizational  Course  - 
utilizes  two  panels: 

CNI  (Navigation) 

CNI  (Conmunications) 


Aircraft  Mechanic  Organizational  Course  - 
utilizes  one  panel : 

- Engine  Fuel  System 

Intermediate  Maintenance  Activity  (IMA)  AV-8A 

Air  Data  Computer  IMA  Course  - utilizes 
one  model : 

- Air  Data  Computer 

Engine  Life  Recorder  IMA  Course  - 
utilizes  one  model : 

- Engine  Life  Recorder 

(This  course  will  be  discussed  at  some 
length  in  the  text  of  this  paper.) 

Head-Up  Display  (HUD)  IMA  Course  - 

utilizes  part  of  the  "0"  level  "Head  Up 
Display  System"  model  in  a FAM  fashion 
for  teaching  symbology,  modes  of  opera- 
tion, and  a system  overview. 

INTERMEDIATE  MAINTENANCE  ACTIVITY 

The  system  maintenance  plan  describes 
the  overall  maintenance  structure  for  each 
system  in  the  AV-8A  and  states  which  Line 
Replaceable  Units  (LRU)  exist  in  that  system. 
At  the  Organizational  Level  (0-Level)  main- 
tenance the  technician  is  required  to  perform 
a systems  checkout  to  the  line  replaceable 
level  and  isolates  and  replaces  faulty  LRU's. 
For  each  LRU  an  individual  maintenance  plan 
exists  and  the  Intermediate  Maintenance 
Activity  (IMA)  or  I-Level  maintenance  tech- 
nician is  responsible  for  this  activity. 
Intermediate  maintenance  activity  requires 
that  the  LRU  be  checked,  repaired,  and  tested 
to  the  Sub-Replaceable  Assembly  (SRA)  level 
and  then  the  technician  repairs  the  SRA,  or 
he  identifies  the  faulty  component  and  the 
SRA  is  sent  to  the  mini-lab  for  component 
replacement.  Two  of  the  simulation  models 
listed  above,  the  "Air  Data  Computer"  and  the 
"Engine  Life  Recorder"  are  IMA  models. 

These  two  simulation  models  were  just 
recently  received  by  the  NAMTD  1006:  the 

"Engine  Life  Recorder"  (ELR)  and  the  "Air 
Data  Computer"  (ADC),  and  will  be  used  for 
I-level  training.  At  the  time  of  this  writing 
the  course  had  not  yet  been  completed  for  the 
Air  Data  Computer;  however,  five  classes  have 
used  the  simulation  model  for  the  ELR,  and 
the  sixth  was  in  progress.  Classes  usually 
contain  four  students  each.  Before  the  school 
had  the  simulation  model,  students  received 
lectures  in  operation,  test  and  checkout  pro- 
cedures for  ELR  Intermediate  Maintenance  Ac- 
tivity. However,  no  practical  exercises  were 
available.  Any  practical  knowledge  gained 
had  tj  await  OJT  at  the  squadron  level. 
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The  Engine  Life  Recorder  (ELR)  "pro- 
vides a visible  means  of  indicating  the  used 
life  of  a qas  turbine  engine.  The  recorder 
is  connected  to  the  thermocouples  in  the  jet 
pipe  and  the  voltage  produced  by  the  thermo- 
couples is  used  to  drive  a numerical  display 
in  the  recorder,  the  rate  of  count  being  re- 
lated to  engine  temperature."  (A.P.112G- 
0132-1) 

The  simulation  model  of  the  Engine 
Life  Recorder  is  designed  primarily  to  pro- 
vide a student  with  an  individualized  hands- 
on  medium  for  operating  and  troubleshooting 
the  ELR  as  depicted  on  the  display  panel, 
figure  9 is  a picture  of  the  Engine  Life 
Recorder  display  panel.  It  also  allows  the 
instructor  to  demonstrate  the  unit  to  a group 
of  students.  Through  inclusion  of  applicable 
supporting  test  equipment,  both  normal  and 
malfunction  operations  in  an  Intermediate 
Maintenance  Activity  can  be  simulated.  The 
display  panel  allows  the  student  or  the  in- 
structor to  make  operational  checks,  trouble- 
shoot, fault  isolate,  and  make  simulated 
equipment  adjustments  and  calibration  checks. 
The  panel  provides  views  and  test  points  to 
allow  these  activities.  Simulated  on  the 
panel  is  an  ELR  Test  set,  a power  supply, 
a VOM,  a stop  watch,  and  a DANA  digital  volt- 
meter. Using  the  simulated  ELR  components 
he  can  perform  the  checkout,  test,  calibra- 
tion and  troubleshooting  procedures  speci- 
fied in  the  technical  manual  (A.P.  112G-0132- 
1). 

Especially  important  to  the  learning 
situation  is  the  fact  that  the  student  can 
practice  making  adjustments  to  the  trim  pots 
which  regulate  the  counters.  For  example; 
the  test  procedures  require  that  certain 
standard  conditions  be  established  for  the 
thermocouple  inputs  and  the  technician  must 
regulate  particular  trim  pots  to  bring  the 
count  within  specification.  He  does  this 
using  the  stopwatch  and  observing  the  ELR 
digital  counter  readout.  This  procedure  is 
done  on  the  simulator  just  as  it  would  be  in 
actual  practice. 

Another  advantage  of  the  simulator 
is  that  it  is  programmed  to  inform  the  student 
when  he  errs.  It  tells  him  where  he  can  go 
for  additional  information.  He  is  made  aware 
of  unsafe  practices  such  as  are  warned  against 
in  the  manual.  Because  it  is  simulated  and 
under  program  control  the  test  equipment 
never  needs  calibration.  For  example;  if 
he  uses  the  ohinneter  with  power  applied  to  the 
unit,  he  doesn't  destroy  a VOM  or  DVM. 

In  addition  to  providing  practice 
in  test  and  checkout  procedures  for  the  ELR 
IMA  student  technician  this  model  also  can  de 
pict  the  following  malfunctions; 


- Diode  D24  open 

- V T 9 open 

- Relay  C on  board  3 inoperative 

- Zener  diode  Z20  on  board  3 open 

- Amplifier  A1  on  board  1 fails 

- Trim  pot  RV4  on  LAW  unit  open 

These  malfunctions  are  entered  through  the 
control  console  keyboard  by  simply  pressing 
their  identifier  code  and  hitting  the  enter 
key.  When  that  is  done  all  observable  indi- 
cations on  the  display  panel  (voltage  checks, 
counts,  etc.)  react  as  they  would  on  the 
actual  equipment.  The  mal functions  pro- 
grammed for  this  simulation  model  were  selec- 
ted on  the  basis  of  their  commonality  and 
frequency  of  occurrence  determined  at  the 
overhaul  facility. 

Thus,  when  the  student  completes  his 
training  using  the  simulation  model,  he  has 
already  had  practice  in  going  through  a full 
checkout  procedure  with  equipment  replicated 
as  being  in  the  normal  state  and  checkout 
procedures  where  various  malfunction^  occur 
during  the  checkout.  He  gets  real,  life- 
like practice  in  "tweaking"  the  ELR  to  bring 
it  into  specification  and  he  also  simulates 
replacement  of  various  representative  compo- 
nents. He  could  not  get  this  with  actual 
equipment  in  the  lab.  Thus,  when  he  leaves 
the  school  for  an  assigned  squadron,  he 
already  has  a basic  useable  maintenance  skill. 

CONCLUSION 

In  its  Organizational  and  some  Inter- 
mediate Maintenance  Activity  training  the 
school  relies  almost  entirely  on  use  of  its 
simulation  models.  When  one  tours  the  NAMTD 
1006  Training  Facility,  there  is  no  test 
or  calibration  equipment  in  the  classes  usinc 
the  simulators. 

The  NAMTD  1006  had  a critical  training 
problem:  To  prepare  maintenance  trainees  for 

the  squadron,  but  they  were  wi tnout  the  nec- 
essary facilities.  In  order  to  meet  their 
objective  they  took  a pragmatic  look  at  what 
was  available  to  solve  their  problem.  They 
made  their  decision  to  select  simulation 
equipment  and  followed  through  with  it. 

Evaluation  of  the  simulators  was  not 
their  intent  - preparing  their  students  to 
be  as  job  ready  as  possible  was,  and  that  is 
what  they  did.  Their  graduates  can  actually 
perform  when  they  arrive  at  their  assigned 
squadrons . 
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Figure  6.  AV-8A  Centralized  Warning  System  Oisplay  Panel 
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Figure  7.  AV-8A  Engine  Fuel  System  Display  Panel 


Figure  8.  Air  Data  Computer  Unit  Display  Panel 
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gure  9.  Engine  Life  Recorder  Unit  Display  Panel 
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INTRODUCTION 

This  paper  is  part  of  work  done  for  the 
National  Science  Foundation  under  Grant  GK 
42071  titled  "Investigation  of  the  application 
of  a hardware  parsed  recursive  string  process- 
ing language  to  graphical  systems."  The  aim 
of  this  grant  is  to  firmware  parse  a certain 
class  of  input  strings  in  the  development  of 
an  extensible  string  processing  language  sim- 
ilar to  text  reckoning  and  compiling  (TRAC).l 
During  this  grant,  a full  operating  language 
called  TOSCL  was  developed  to  run  on  a Data 
General  NOVA  1220.2-3  The  second  phase  was 
to  implement  the  parse  on  the  Intel  3000  sys- 
tem. 


The  emphasis  in  design  and  construction 
of  the  TOSCL  microprogramming  system  was  to 
be  able  to  microprogram  with  the  least  amount 
of  effort.  Since  the  major  programming 
effort  was  to  be  the  development  of  TOSCL, 
only  a minimum  of  software  was  developed. 

The  minimum  support  required  was  (1)  a sym- 
bolic editor,  (2)  a microcode  assembler, 

(3)  a loader  for  microcontrol  store,  and 

(4)  microprogramming  debugging  aides.  This 
paper  shows  how  these  programs  combined  with 
hardware  functions  allowed  simulation  and 
debugging  of  the  Intel  3000  microprocessor 
system. 

THE  MICR0C0NTR0L  WORD 


The  primary  aim  of  this  paper  is  to  de- 
scribe the  implementation  of  microprograms 
using  a sixteen  bit  minicomputer  and  to  show 
some  of  the  software  developed  to  support 
microcoding,  assembling  of  microcode,  loading 
the  microcontrol  store,  and  the  debugging  of 
the  microprograms. 

The  development  of  Intel's  Schottky 
Bipolar  LSI  microcomputer  elements  has  brought 
microprogramming  to  the  field  of  microproces- 
sors. These  devices  allow  almost  infinite 
variety  of  design  applications,  control  word 
size,  and  control  word  configuration.  This 
versatility  makes  the  development  of  support 
software  a difficult  if  not  impossible  task. 


The  minicomputer  supporting  the  TOSCL 
system  is  a Data  General  NOVA  1220,  and 
since  this  is  a sixteen  bit  machine,  a 
control  store  size  of  sixteen  bits  is 
desirable.  The  3000  system  configuration 
suggested  by  Intel  uses  an  eighteen-plus 
bit  control  word  (see  Figure  la).  To  imple- 
ment a sixteen  bit  control  word,  it  was 
necessary  to  give  up  some  flexibility  in  the 
system.  After  looking  into  the  application 
which  is  string  interpretation,  the  facil- 
ities considered  least  critical  are  portions 
of  the  flag  control  and  branching  (jumpcode) 
facilities. 
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Figure  1.  Microcontrol  Word 
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The  resulting  control  word  is  sixteen  bits 
(see  Figure  lb).  It  represents  a loss  of  four 
jump  instructions  and  individual  control  of 
one  flag  bit,  the  Z-flag  bit.  The  K-buss  mask 
can  only  be  set  to  all  one  or  all  zero. 

The  most  often  used  3000  branch  comnand 
uses  the  PX  inputs  of  the  3000  system  to 
force  a jump  in  control  store.  Since  this 
major  decision  process  is  accomplished 
through  a hardware  decode,  the  use  of  one 
flag  control  bit  is  all  that  is  necessary.  A 
reduction  in  the  jump  set  could  be  accomplish- 
ed with  the  loss  of  the  use  of  one  of  the  flag 
functions.  Random  logic  has  been  added  to 
accommodate  this  change  in  the  control  word 
format.  The  3000  control  word  jump  instruc- 
tion set  is  hardware  decoded  to  recover  the 
bits  removed  by  the  change  in  the  control 
word.  The  1/0  signals  are  also  decoded  from 
the  NOP  Instructions  and  its  operand  (func- 
tion field)  to  accomplish  the  required  input, 
output,  and  memory  control. 

The  resulting  control  word  is  sixteen 
bits  (see  Figure  lb).  It  represents  a loss 
of  four  jump  instructions  and  individual 
control  of  the  flags. 

THE  ASSEMBLER 

The  Intel  3000  system  allows  such  a 
variety  of  configurations  that  the  resulting 
generalized  assemblers  are  often  difficult 
to  use,  hard  to  transfer  between  machines, 
impossible  to  implement  on  small  systems, 
and  require  large  amounts  of  computer  time. 

The  concept  developed  was  not  to  generate 
a general  assembler  but  to  use  an  existing 
assembler  with  macro  capabilities  to  define 
an  instruction  set  which  includes  both  Intel 
instructions  and  instructions  unique  to  this 
task.  This  loss  of  portability  due  to  the 
omission  of  four  jump  instructions  and  the 
control  of  the  Z-flag  was  examined  and  found 
to  be  not  sufficient  to  warrant  additional 
efforts.  This  use  of  a macroassembler  to 
generate  a cross-assembler  through  the  defini- 
tion of  a set  of  instructions  is  an  efficient 
approach  but  results  in  a highly  machine  de- 
pendent cross-assembler. 

The  cross-assembler  developed  to  assemble 
programs  for  the  T0SCL  system  is  run  on  Data 
General's  macroassembler  and  has  the  follow- 
ing format: 

(LABLE:)  0PC0DE(H)  (1)  OPERAND  JUMPC0DE 
DISPLACEMENT 

The  portion  shown  in  parenthesis  is  op- 
tional. The  H appended  to  the  opcode  con- 
trols the  hold  of  the  carry  flag.  The  1 is 
used  to  set  the  flag  output  (F0)  which  is 
used  as  the  carry  input  (Cl).  The  label  and 


comment  fields  are  aides  for  the  pro- 
grammer to  use  in  documentation. 

The  operand  field  specifies  the  range 
of  the  opcode  and  the  K-buss  information. 

The  operands  are  broken  down  into  three 
register  groups:  (1)  register  operations, 

(2)  memory  operations,  and  (3)  input  opera- 
tions . 

The  jumpcode  and  displacement  are  en- 
tirely independent  of  the  opcode  and  op- 
erand. Therefore,  the  programming  tech- 
niques for  this  system  are  based  on  estab- 
lished microprogramming  techniques. 

The  output  from  the  macroassembler  is 
in  a special  binary  format.  This  format 
can  be  reduced  to  a core  image  file  using 
the  relocatable  loader.  This  software  along 
with  the  symbolic  editor  is  part  of  the 
standard  system  and  requires  no  modifica- 
tion. The  resulting  core  image  file  can 
either  be  executed  from  the  host  systems 
memory  or  can  be  loaded  into  the  control 
store  (high-speed  random  access  memory 
(RAM))  that  appears  to  the  3000  system  as 
a read  only  memory  (ROM). 

T0SCL  SYSTEM  HARDWARE 

The  current  design  of  the  Intel  3000 
system  used  in  the  T0SCL  System  is  shown 
in  Figure  2.  This  system  uses  Intel's 
suggested  system  architecture. 5, 6 The 
RAM  for  the  3000  system  is  the  host  system 
memory  and  is  accessed  using  direct  memory 
access  (DMA).  This  memory  is  addressed 
using  the  A-outputs  of  the  CPE  array,  while 
the  D-outputs  are  used  for  the  write  data, 
and  the  M- inputs  are  used  to  read  memory. 

The  DMA  operation  is  automatically  initi- 
ated by  decoding  certain  3000  commands 
(LMM,  LMI , LMMH,  and  LMMIH).  If  a previous 
DMA  request  has  not  been  completed,  the 
instruction  will  be  deferred  by  halting 
the  clock  until  the  operation  is  complete. 
Also  to  be  sure  that  the  microprogram  will 
not  try  to  use  DMA  results  prematurely,  the 
system  has  a wait-for-memory  command  (WAIT). 
The  system  allows  for  computation  that  does 
not  effect  significant  registers  to  be 
performed  while  the  DMA  operations  are 
being  completed.  This  may  take  from  1500 
to  3500  microseconds. 

The  clock  circuit  can  be  run  in  two 
basic  modes,  free  running  and  single  step. 
The  free  running  mode  once  enabled  will  run 
until  it  is  halted.  The  halt  can  be  gen- 
erated by  one  of  two  commands.  Done  sets 
the  done  flag  of  the  host  system,  and  INTR 
generates  a software  interrupt  to  the  host 
system.  The  clock  is  also  temporarily 
halted  by  the  second  DMA  request  until  the 
first  has  been  completed.  The  second  mode 
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of  the  clock  is  single  step.  This  allows  for 
the  execution  of  one  microinstruction  at  a 
time  and  is  used  for  tracing  and  debugging 
programs.  The  system,  when  in  single  step 
mode,  uses  the  main  memory  as  the  source  for 
the  microinstructions.  For  normal  operation, 
the  control  store  memory  (256  x 16  bits)  is 
used  as  the  source  of  microinstructions. 

This  memory  can  be  loaded  directly  from  a 
core  image  file  which  was  generated  by  the 
cross-assembler  and  the  relocatable  loader. 


The  contents  of  the  control  store  memory 
can  be  examined,  modified,  or  verified 
under  program  control. 

The  local  memory  (16  x 16  bits  RAM)  is 
used  to  jam  instructions  into  the  3000  sys- 
tem. Through  the  jaming  of  the  proper 
instructions,  all  of  the  registers  may  be 
examined.  Also  the  control  store  address 
may  be  examined,  along  with  the  contents  of 
control  store. 


DATA  GENERAL  1/0  BUSS 


•TRI-STATE 


Figure  2.  TOSCL  System  Diagram 
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Figure  3.  Jump  Set  and  Decode 


The  jump  decode  operation  is  shown  in 
Figure  3,  above.  The  resulting  jump  set,  as 
has  been  discussed,  is  reduced  from  the  orig- 
inal Intel  jump  set,  but  was  found  to  accom- 
plish the  desired  operations  with  no  dif- 
ficulty. 

The  JPX  encode  is  used  to  detect  and 
encode  privileged  characters.  The  input  to 
the  JPX  encode  is  the  M register  which  is  the 
Memory  Buffer  Register  of  the  TOSCL  system. 
The  word  in  the  M register  is  then  encoded 
and  used  to  force  a jump  in  control  store 
based  on  the  character  in  the  M register.  The 
M register  also  can  be  used  to  force  the 
first  address  in  control  store  through  the 
use  of  the  LOAD  input  of  the  memory  control 
unit. 

The  system  has  inputs  to  and  from  the 
host  computer  which  are  used  for  the  Simula- 
tion  of  the  final  system.  The  final  system 
would  also  use  its  own  memory  thus  speeding 
overall  system  operation. 

SUMMARY  AND  COMMENTS 

The  development  of  microprocessing  system 
simulators  has  been  typically  based  on  a 
cross-assembler  and  simulator  package  design- 
ed to  run  on  a large  computer  system.  The 
program  developed  on  the  simulator  is  then 
loaded  into  a separate  microprocessor  based 
system  which  is  used  to  support  the  hardware 
design  phase. 

The  effort  at  Florida  Technological  Uni- 
versity has  resulted  in  an  integrated  soft- 
ware and  hardware  support  system.  It  uses 
standard  minicomputer  software  (macroassem- 
bler, loader,  editor,  etc.)  and  Interfacing 
facilities  to  provide  not  only  a powerful 
development  phase  tool  but  also  a base  for 
evaluation  and  training. 


Evaluation  can  be  accomplished  by 
adding  hardware  to  gather  run  time  statis- 
tics which  can  then  be  processed  b>  the 


minicomputer  system.  Training  can  be 
accomplished  using  a combination  of  computer- 
aided  instruction  and  real-time  operation. 
Both  of  these  "side  effects"  are  presently 
being  investigated. 

One  final  comment  on  the  technique 
used  for  microprocessor  system  simulation; 
very  often  a great  deal  of  cost  and/or 
effort  is  spent  on  obtaining  design  and 
development  tools  which  are  essentially  a 
one-time  effort.  If  a minicomputer  is 
available,  it  can  be  used  as  a base  for 
software  development  aimed  at  a general 
system  that  can  be  used  with  the  micro- 
processor that  is  best  suited  for  the  in- 
dividual application.  It  appears  that  this 
approach  is  the  straightforward  and  cost- 
effective  means  of  system  design. 
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ABSTRACT 

The  National  Aeronautics  and  Space  Ad- 
ministration (NASA)  is  currently  de- 
veloping an  economical  space  transpor- 
tation system  known  as  the  Space  Shut- 
tle. Various  ground  support  equipment 
(GSE)  has  to  perform  critical  func- 
tions to  assure  a successful  launch  of 
the  Shuttle  vehicle.  In  order  to  test 
this  equipment  under  simulated  launch 
conditions,  NASA  is  activating  a 
Launch  Equipment  Test  Facility  (LETF). 
This  facility  will  simulate  effects 
such  as  vehicle  deflections  and  oscil- 
lations at  the  pad,  rain,  cryogenic 
shrinkage,  vehicle  lift-off,  and  solar 
heating.  A physical  description  of 
the  LETF  and  its  cost-effective  utili- 
zation is  provided  in  this  paper. 

Test  articles,  test  categories,  and 
test  descriptions  are  detailed.  Par- 
ticular emphasis  is  placed  on  simula- 
tion equipment  and  conditions. 

INTRODUCTION 

Following  the  successful  completion  of 
the  Apollo  Program,  construction  ef- 
forts on  the  Space  Shuttle  launch  and 
landing  project  started  in  1974.  The 
intended  purpose  of  the  Space  Shuttle 
Program  is  to  provide  an  economical 
space  transportation  system  to  carry 
men,  equipment,  laboratories,  satel- 
lites, and  propulsion  stages  to  and 
from  Earth  orbit.  This  program  is  ex- 
pected to  reduce  the  cost  of  space 
transportation  to  a fraction  of  the 
cost  of  present  expendable  booster  ve- 
hicles. Figure  1 shows  the  Shuttle 
process i ng- 1 aunch- 1 and  i ng  cycle. 

Launching  a Space  Shuttle  Vehicle 
(SSV)  involves  synchronized  operation 
of  launch  critical  GSE  and  highly 
trained  launch  personnel.  The  Space 
Shuttle  Program  requires  testing  of 
launch  critical  GSE  under  every  possi- 
ble condition.  To  provide  a testing 
environment  for  the  equipment,  NASA 
has  designed  and  developed  the  LETF  at 
John  F.  Kennedy  Space  Center  (KSC), 
Florida,  which  will  be  used  to  test 
GSE  under  various  simulated  launch 
condi t i ons  . 

The  ability  of  the  GSE  to  react  pro- 
perly under  various  conditions  must  be 


verified  prior  to  committing  the  equip- 
ment to  a manned  launch.  The  LETF  will 
be  used  to  test  GSE  which  could  cause 
failure  to  the  vehicle  if  it  did  not 
function  properly.  Various  launch  ve- 
hicle conditions  such  as  vehicle  de- 
flections and  oscillations  at  the  pad, 
rain,  solar  heating,  cryogenic  shrink- 
age, and  vehicle  lift-off  will  be  si- 
mulated by  the  facility. 

FACILITY  OBJECTIVES 

The  primary  objective  of  the  LETF  is 
to  provide  a capability  for  qualifying 
and  certifying  operability,  reliabili- 
ty, and  maintainability  of  launch  cri- 
tical GSE  prior  to  installation  at  the 
launch  complex  or  between  launches 
when  recertification  is  required.  A 
secondary  objective  is  to  provide  a 
ipability  for  development  testing  of 
conceptual  GSE  designs. 

JUSTIFICATION 

Previous  investigations  indicated  the 
SSV  schedule  could  be  affected  by 
needlessly  shipping  the  GSE  back  and 
forth  to  George  C.  Marshall  Space 
Flight  Center  (MSFC)  at  Huntsville, 
Alabama,  for  testing.  Using  the  sal- 
vaged Saturn  V test  equipment  from 
MSFC  and  erecting  this  same  equipment, 
modified  to  Shuttle  requirements,  at 
KSC  was  determined  to  be  advantageous 
to  the  Space  Shuttle  Program  in  terms 
of  time,  cost,  and  proximity  to  the 
launch  complex.  Hence,  it  was  decided 
to  design  and  develop  the  LETF  to  sup- 
port tests  on  access  arms,  umbilicals, 
and  associated  GSE  at  KSC. 

CRITICAL  GSE  TEST  ARTICLES 

The  LETF  will  provide  the  capability 
for  testing,  qualifying,  and  certify- 
ing the  operation  and  reliability  of 
the  Orbiter  access  arm  (OAA),  tail 
service  masts  (TSMs),  external  tank 
(ET)  gaseous  hydrogen  (GH2)  vent  umbi- 
lical and  access  arm,  and  solid  rocket 
booster  (SRB)  support/holddown  posts. 
Such  equipment  is  considered  launch 
critical  because  their  malfunction  can 
cause  damage  to  the  SSV  as  well  as  a 
mission  failure. 
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OAA.  The  Shuttle  OAA  is  a modified 
Apol 1 o swing  arm  #9.  It  provides  ac- 
cess and  egress  between  the  Space 
Shuttle  access  tower  and  the  Orbiter 
vehicle.  An  environmental  chamber  is 
located  at  the  end  which  interfaces 
wi th  the  Orbi ter. 

TSMs . The  TSMs  provide  the  capability 
for  1 oa di ng/of f 1 oad  i ng  liquid  oxygen 
( L 0 X ) and  liquid  hydrogen  ( L H 2 ) into 
the  SSV.  During  lift-off,  the  TSMs 
disconnect  and  retract  the  T-0  umbili- 
cals  from  the  aft  end  of  the  Orbiter 
and  store/protect  them  within  the  TSM 
blast  housings  from  the  high  tempera- 
ture exhaust  plume  and  blast  loads  of 
the  SSV. 

ET  GH2  Vent  Umbilical  and  Access  Arm. 
This  system  allows  venting  of  GH2  from 
the  ET  and  achieves  umbilical  discon- 
nect, withdrawal,  and  line  management 
as  the  SSV  ET  ascends  at  launch. 

SRB  Support /Holddown  Posts  . Eight 
support/holddown  posts  will  be  used 
for  supporting,  restraining,  and  re- 
leasing the  SSV  SRBs  during  launch. 
These  posts  also  provide  jacking  and 
leveling  capability  for  stacking  the 
SRBs. 

TEST  CATEGORIES 

At  KSC,  three  test  categories  have 
been  established  to  separate  the  test- 
ing of  Space  Shuttle  launch  critical 
GSE . 

Concept  Verification  Tests.  Concept 
verification  tests  consist  of  a series 
of  tests  performed  to  verify  basic 
theoretical  or  experimental  design 
principles  and  concepts.  The  tests 
are  designed  to  evaluate  the  design 
concepts  of  critical  parts  under  simu- 
lated conditions  of  operation;  confirm 
adequacy  of  selection  of  components 
and  parts;  and  determine  capabilities 
of  equipment  under  test.  Concept  veri- 
fication tests  have  already  been  ac- 
complished and  successfully  completed 
on  TSM  and  ET  GH2  vent  umbilical  test 
articles. 

Prototype  Tests.  Prototype  tests  con- 
sist of  a series  of  tests  performed  on 
certain  test  articles  to  demonstrate 
physically  its  dynamic  function,  suit- 
ability, and  reliability.  Data  is  de- 
veloped from  these  tests  which  provide 
the  means  to  estimate  the  confidence 
level  of  the  system  to  perform  its  In- 
tended function.  If  acceptable,  these 
prototype  test  articles  may  become 
flight  items.  A prototype  model  of 


the  TSM  has  been  designed,  fabricated, 
and  assembled  and  will  be  tested  in 
the  fall  of  1976. 

Qualification  Tests.  Qualification 
tests  consist  of  a series  of  tests 
performed  on  actual  flight  configured 
hardware  to  demonstrate  that  the  de- 
sign criteria  have  been  satisfied  and 
that  the  items,  as  produced,  will  per- 
form the  required  functions  reliably 
in  accordance  with  the  test  criteria. 

IMPLEMENTATION  PHASES 

The  LETF  project  is  being  accomplished 
under  three  phases  (Ref.  1). 

Phase  I Implementation.  Phase  I im- 
plementation includes  planning,  devel- 
opment, facility  design,  construction, 
installation  of  basic  facility  equip- 
ment, and  checkout  (Figure  2). 

Phase  II  Implementation.  Phase  II  im- 
plementation (Figure  3)  is  being  ac- 
complished under  a combined  phase  1 1 A 
and  IIB  effort.  Phase  1 1 A implementa- 
tion entails  planning,  development, 
support  systems  design,  construction, 
installation  of  simulation  systems  and 
other  GSE,  and  checkout.  Phase  IIB 
implementation  is  a parallel  effort  to 
phase  1 1 A which  includes  refurbishing 
and  modifying  the  OAA  random  motion 
simulator  (RMS),  TSM  and  ET  GH2  vent 
line  RMS  and  lift-off  simulator  (LOS), 
and  SRB  support/holddown  posttest 
fixture.  Upon  conducting  a successful 
checkout  of  the  test  facility  under 
phase  II,  the  LETF  will  be  ready  for 
launch  critical  GSE  testing. 

Phase  III  (Operational)  . Phase  III 
(Figure  4)  includes  ins  t a 1 1 a r.  i o n , 
checkout,  and  test  of  the  four  launch 
critical  GSE  test  articles,  i.e.,  OAA, 
TSMs,  ET  GH2  vent  umbilical  and  access 
arm,  and  SRB  support/holddown  posts. 
These  test  articles  are  presently  be- 
ing designed,  fabricated,  and  assem- 
bled at  KSC. 

LETF  DESCRIPTION 

Significant  items  that  comprise  the 
LETF  are  briefly  discussed  in  the  fol- 
lowing paragraphs  (Ref.  2).  Refer  to 
Figures  2 and  3 for  relative  locations 
in  the  LETF  of  each  of  the  items  dis- 
cussed . 

SRB  Support/Holddown  Posttest  Fixture 
Pad . This  pad  will  support  the  SRB 
support/holddown  posttest  fixture. 

OAA  RMS  Pad.  This  pad  will  accommodate 
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the  test  equipment  which  consists  of 
an  RMS  and  an  Orb  iter  skin  section. 

The  OAA  test  will  be  performed  on  this 
pa<}. 

TSM  and  ET  GH2  Vent  Line  and  Access 
Krm  RMS  Pad.  This  pad  will  be  used 
for  the  T$M  test  and  the  ET  GH2  vent 
umbilical  and  access  arm  test.  A 
platform  is  provided  to  support  the 
simulator  for  simultaneously  testing 
the  umbilical  and  access  arm  and  the 
TSM.  Figure  5 shows  the  platform  on 
the  pad.  The  tower  simulator  is  shown 
in  the  background. 

T ower  Simula  tor . The  tower  simulator 
Ta  three-level  section  of  existing 
launch  umbilical  tower  #3  approximate- 
ly 60  ft  in  height)  is  supported  on 
four  footings.  The  area  immediately 
under  the  tower  houses  the  hydraulic 
pumps  for  the  RMSs  and  other  related 
GSE.  Figure  6 shows  the  tower  simula- 
tor, hydraulic  pump  house,  and  trans- 
f ormers  . 

General  Pads.  Eighteen  small  general 
purpose  pads  measuring  approximately 
Z ft  by  2 ft  have  been  placed  as  re- 
quired for  electrical  cabling,  hydrau- 
lic, and  pneumatic  line  support. 

Pump  House.  The  pump  house  (Figure  6), 
located  at  the  base  of  the  tower  simu- 
lator, includes  four  100-gpm  motors/ 
pumps,  hydraulic  oil  reservoir,  hy- 
draulic charging  unit,  an  exhaust  fan, 
and  other  ancillary  equipment. 

LH2  System.  Liquid  H2,  supplied  from 
LH2  tankers,  will  be  used  to  chill 
down  to  cryogenic  temperatures  the  TSM 
LH2  umbilical  and  lines.  Hydrogen  gas 
will  be  vented  through  the  GH2  vent 
umbilical  and  also  to  the  atmosphere. 
All  LH2  lines  are  vacuum  jacketed. 

LN2  System.  Liquid  N2,  supplied  from 
LN2  tankers , will  be  used  to  simulate 
LOX  flowing  through  the  LOX  umbilical 
for  pressurization. 

GN2_Sys tern . Gaseous  N2,  supplied  from 
10,000  psi  GN2  tube  trailers,  will  be 
used  for  purging  umbilicals,  opera- 
tional intercommunication  system  com- 
munication boxes,  ac  power  distribu- 
tors, and  as  a media  for  system  pres- 
surization. 

GHe  System.  Gaseous  He,  supplied  from 
GHe  tube  trailers,  will  be  used  for 
purging  GH2  vent  umbilical  and  LH2  TSM 
umb i 1 1 ca  1 . 

AC  Power  Distribution  System . This 


system  is  supplied  by  a 13.2  kV/480  V 
750  kVA  transformer  and  primary  switch 
Secondary  voltages  of  480  V and  277/, 
115  V will  be  distributed  to  the  pump 
house,  service  tower,  and  test  simula- 
tors to  satisfy  ac  power  and  lighting 
requirements  within  the  test  area.  AC 
power  is  provided  to  the  test  control 
room  from  existing  power  panels  in 
Building  M7-505  . 

Control  and  Monitor  Sy s tern . The  con- 
trol  and  monitor  system  is  capable  of 
remote  control  from  the  test  control 
room  (Figure  7)  during  the  ET  GH2  vent 
umbilical  and  TSM  tests  ( H 2 flowing). 
Local  control  will  be  provided  at  the 
simulator  areas  (H2  not  flowing). 

Cabling.  Cabling  is  routed  in  trench- 
es covered  with  steel  grating  (Figure 
8)  from  the  pump  house  to  the  control 
room  and  simulators.  All  ac  power 
cabling  within  the  trenches  is  sealed 
in  high  voltage  conduit. 

I ns trumen ta  t i on . Approximately  350 
measurements  will  be  needed  to  acquire 
data  for  functional,  qualification, 
and  final  verification  of  GSE/flight 
hardware  interfaces  during  simulated 
tests.  Instrumentation  test  equipment 
will  include  recorders,  transducers, 
strain  gages,  load  cells,  optical  in- 
strumentation, flow  meters,  and  other 
necessary  sensors  to  fulfill  all  spec- 
ified conditions  of  usage.  Readings 
will  be  analog,  digital,  and/or  actual 
gage  readouts.  These  outputs  will  be 
recorded  for  complete  evaluation  of 
all  tests  required  to  support  this 
phase  of  the  Shuttle  Program. 

Operati ona  1 Intercommunication  System . 
An  operational  i ntercommuni cation  sys- 
tem will  be  provided  at  the  IETF.  The 
system  will  be  connected  intc  the  main 
system  already  installed  in  the  Opera- 
tions and  Checkout  Building  and  to  the 
test  stands. 

TEST  SIMULATORS  AND  FIXTURES 

The  simulation  test  equipment  was  pre- 
viously located  at  MSFC,  where  it  was 
used  in  the  Apollo  Program  for  GSE 
testing.  The  equipment  was  brought  by 
barge  in  March  1975  to  KSC,  where  the 
simulators  were  cost-effectively  re- 
furbished and  modified  for  the  Space 
Shuttle  configuration. 

From  calculations  based  on  previous 
launch  experience,  the  motion  of  the 
Orbiter  before  launch  due  to  various 
effects  such  as  wind  deflections,  so- 
lar heating,  cryogenics,  and  engine 
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serious  hardware  damage. 

Upon  successful  completion  of  manual 
testing,  the  ability  of  the  umbilical 
carrier  to  track  the  simulated  Orbiter 
vehicle  motions  will  be  verified. 

This  series  of  tests  will  be  conducted 
with  the  umbilical  carrier  mated  to 
the  Orbiter  vehicle  RMS.  All  relative 
motions  between  the  Orbiter  vehicle 
and  the  TSMs  will  be  simulated,  in- 
cluding the  full  range  of  Orbiter  ve- 
hicle oscillation  frequencies  and  sim- 
ulated "vehicle  fueled"  and  "vehicle 
unfueled"  conditions.  The  TSM  system 
will  be  monitored  throughout  these 
tests  for  binding,  chafing,  and  loos- 
ening of  components. 

The  next  series  of  tests  will  verify 
proper  operation  of  the  TSM  system 
during  lift-off  simulation.  The  TSM 
system  will  be  placed  in  actual  launch 
configuration  and  operated  automati- 
cally during  each  test  cycle.  Early 
tests  in  this  series  (with  no  vehicle 
random  motion)  will  be  conducted  with 
the  Orbiter  vehicle  LOS  placed  in  a 
nominal,  "vehicle  fueled"  position 
relative  to  the  TSM.  Maximum  and  min- 
imum vehicle  lift-off  accelerations 
will  be  simulated.  Tests  will  be  con- 
ducted with  the  Orbiter  vehicle  LOS 
placed  in  anticipated  "extreme"  posi- 
tions with  respect  to  the  TSM.  Fol- 
lowing these  tests,  random  motion  of 
the  Orbiter  vehicle  simulator,  at  both 
"nominal"  and  "extreme”  positions, 
will  be  introduced  with  lift-off  oc- 
curring in  conjunction  with  random  mo- 
tion. As  this  phase  of  testing  pro- 
gresses, cryogenic  lines  will  be 
chilled  (using  LH2  and  L N 2 ) prior  to 
lift-off  simulation.  This  series  of 
tests  will  be  completed  using  chilled 
lines  and  simulated  rain  conditions. 

The  final  series  of  tests  will  be  sim- 
ilar to  those  described  in  the  pre- 
vious paragraph  but  with  the  added 
condition  of  simulated  system  failures. 
Selected  portions  of  the  umbilicals 
and  mast  disconnect/retract  systems 
will  be  disabled  prior  to  vehicle 
lift-off  simulation. 

ET  GH2  Vent  Umbilical  and  Access  Arm 
Tes  t . TeTtTng  of~TFe~  ET  6H 2 vent  um- 
bTTTcal  and  access  arm  system  will 
commence  with  proof  load  tests  of  the 
arm  structure  and  measurement  of  arm 
deflections.  Concurrent  with  these 
tests,  loads  imposed  on  the  fixed-arm 
platform  and  tower  simulator  structure 
will  be  determined. 

Manual  operation  of  the  arm  will  be 


performed  to  verify  interfaces  and 
mechanical  integrity  of  the  arm  compo- 
nents. A series  of  tests  consisting 
of  alignment  and  adjustment  of  arm 
components  and  arm  positioning  stops 
will  be  performed.  Extension  and  re- 
traction of  the  arm  to  verify  clear- 
ances and  interfaces,  freedom  from 
binding,  as  well  as  locking  and  latch- 
back  verification  will  be  accomplished. 
The  ET  skin  panel  with  its  umbilical 
on  the  LOS  structure  coupled  with  the 
umbilical  system  (ground  side)  at- 
tached to  arm  cradles  will  be  cycled 
throughout  its  expected  motion  enve- 
lope and  monitored  for  satisfactory 
operation.  Umbilical  system  operation 
(under  reguired  motion,  including 
launch  abort)  with  all  systems  func- 
tioning, including  GH2  chi  lid own  and 
simulated  rain,  will  be  conducted  to 
verify  satisfactory  operation.  Dynam- 
ic operation  of  the  system  normal  dis- 
connect and  disconnect  in  the  back-up 
mode  will  be  accomplished  in  the  same 
manner  as  previously  described. 

SRB  Support/Hol  ddown  posttest.  These 
tests  will  be  conducted  to  verify  the 
capability  of  the  SRB  support/holddown 
posts  to  support,  rest  rain,  and  re- 
lease safely  and  reliably  the  SSV  dur- 
ing prelaunch  and  launch  operations. 

The  SRB  support/hol ddown  posttests 
will  be  basically  structural  in  na- 
ture. Testing  under  down-loading,  up- 
loading, side-loading,  and  combined- 
loading  will  be  accomd i shed  . Stress 
readings  and  post-deflections  will  be 
monitored  and  recorded.  Torque  and 
preload  tests  of  nut  and  bolt  will  be 
accomplished  to  determine  optimization. 

Explosive  release  synchroni zation 
tests  will  be  conducted  to  verify 
firing  of  release  nuts  withir  the  pre- 
scribed time  prior  to  simulated  SRB 
ignition  and  release  command.  Timing 
of  release  will  be  verified.  Certain 
component  failures  will  be  simulated 
to  test  the  reliability  of  the  system. 

CONCLUSIONS 

The  Launch  Equipment  Test  Facility  at 
KSC  provides  a cost-effective  means 
for  qualification  testing  of  launch 
critical  GSE  for  the  Space  Shuttle 
Program.  Use  of  refurbished  and  modi- 
fied simulation  test  equipment  from 
the  Apollo  Program  has  provided  con- 
siderable cost  savings. 

System  countdown  and  test  procedures 
will  be  prepared  for  LETF  testing 
scheduled  to  commence  in  the  Spring  of 


ignition  will  have  been  predicted. 

The  TSMs,  OAA , and  ET  vent  line  will 
be  tested  in  the  LETF  to  ascertain  the 
capability  of  these  equipments  to  sur- 
vive such  motion.  The  simulators  used 
for  this  function  consist  mainly  of 
two  basic  functions,  i.e.,  random  mo- 
tion simulation  and  lift-off  simula- 
tion. Random  motion  simulation  is  re- 
quired for  testing  the  TSM,  OAA , and 
ET  vent  line.  Testing  the  SRB  sup- 
port/holddown posts  will  be  accom- 
plished using  a special  test  fixture 
designed  for  that  purpose. 

Random  Motion  and  Lift-off  Simulators. 
A typical  RMS  and  LOS  (Figure  9 ) con- 
sist  of  a servo-controlled  hydraulic/ 
pneumatic  resonant  drive  system,  lu- 
brication and  cooling  systems,  and  a 
control  system.  Random  motion  in  two 
orthogonal  directions  in  the  horizon- 
tal plane  is  simulated  by  two  car- 
riages capable  of  generating  Lissajous 
figures  ranging  from  a straight  line 
to  a circle.  Vertical  motion  is  simu- 
lated by  the  LOS.  Friction  is  mini- 
mized by  using  a rail  and  roller  de- 
sign. 

The  LOS  consists  of  a tower  structure 
in  which  two  elevator  panels  ride  ver- 
tically up  and  down  to  simulate  the 
lift-off  conditions.  Each  panel  can 
be  controlled  independently  with  its 
hydraulic  drive  system.  The  panels 
are  used  for  the  two  TSMs.  The  OAA 
flight  skin  panel  is  rigidly  mounted 
on  the  modified  tower  structure  which 
is  attached  to  the  RMS.  It  has  no 
vertical  mot’ on  capabilities. 

SRB  Support/Holddown  Posttest  Fixture. 
This  is  a structura 1 fixture  which  si- 
mulates SRB  loads.  It  uses  two  hy- 
draulic cylinders  to  apply  test  loads 
to  the  SRB  support/holddown  post  in 
two  orthogonal  directions.  These  cy- 
linders will  be  used  separately  and  in 
combination  to  test  the  loading  capa- 
city of  the  holddown  post. 

TEST  CONDITIONS 

As  required  by  the  Shuttle  Master 
Verification  Plan  (Ref.  3),  testing 
will  be  performed  under  conditions  si- 
mulating vehicle  motion  before  launch, 
fueling  and  purging,  system  power-up/ 
power-down,  emergency,  hold,  and  other 
si tuations . 

Launch  environment  is  affected  consid- 
erably by  rain,  solar  heating,  cryo- 
genic effects,  Orbiter  deflections  and 
oscillations,  and  vehicle  lift-off. 
Satisfactory  operation  of  the  launch 


critical  6SE  has  to  be  insured  under 
such  a wide  range  of  conditions.  The 
LETF  will  simulate  these  effects  and 
will  be  used  to  investigate  the  re- 
sponse of  GSE  under  these  simulated 
launch  conditions. 

DESCRIPTION  OF  TESTS 

OAA  Test.  Testing  the  OAA  system  will 
commence  with  proof  load  tests  of  the 
arm  structure  and  measurement  of  arm 
deflections.  Concurrent  with  these 
tests,  loads  imposed  on  the  tower  si- 
mulator structure  will  be  determined. 

A series  of  tests  consisting  of  align- 
ment and  adjustment  of  the  OAA  compo- 
nents will  follow  the  load  tests.  The 
interface  between  the  environmental 
chamber  on  the  arm  and  the  Orbiter 
skin  panel  attached  to  the  RMS  will  be 
checked,  and  the  arm  positioning  stops 
will  be  adjusted  accordingly.  The  op- 
eration and  adjustment  of  the  latch- 
back  mechanism  equipment  on  the  OAA 
and  tower  simulator  will  be  verified. 
Furthermore,  all  components  will  be 
checked  for  free  movement  and  inter- 
ference during  arm  rotation. 

The  OAA  will  be  extended  t(  yard  the 
Orbiter  skin  panel  which  is  attached 
to  the  RMS,  and  the  loads  imposed  on 
the  Orbiter  will  be  determined.  The 
seal  between  the  environmental  chamber 
and  the  Orbiter  skin  panel  will  be 
monitored  for  satisfactory  operation. 

The  operation  of  the  OAA  rotation  sys- 
tem will  be  verified  in  a series  of 
manually  and  automatically  controlled 
tests.  Moving  the  OAA  to  the  extend 
and  retract  positions  will  be  tested 
with  the  Orbiter  skin  panel  in  various 
positions.  Certain  component  failures 
will  be  simulated  to  test  the  reliabil- 
ity of  backup  systems. 

Testing  will  include  verification  of 
the  systems  serving  the  environmental 
chamber,  i.e.,  environmental  controls 
and  electrical/communications  equip- 
ment. In  addition,  rain  conditions 
will  be  simulated  with  and  without  ve- 
hicle motion  simulation. 

TSM  Test . Testing  of  the  TSM  system 
will  commence  with  manually  controlled 
operation  of  the  umbilical  disconnect 
and  retract  mechanisms.  Proper  opera- 
tion of  the  retract  mechanism  will  be 
verified.  These  tests  will  validate 
the  operation  of  the  vari ous  mecha- 
nisms. Since  they  are  conducted  man- 
ually, tests  can  be  quickly  terminated 
in  event  of  malfunction  to  preclude 
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1977.  These  procedures  will  be  used 
by  technicians  and  launch  operations 
personnel  to  perform  the  qualification 
tests.  Testing  and  subsequent  re- 
quired changes /modi fi ca ti ons  will  "de- 
bug" the  systems  and  qualify  them  for 
manned  launches  of  the  SSV.  The 
training  and  experience  acquired  by 
technicians  and  launch  personnel  on 
the  LETF  operation  will  qualify  them 
for  launch  complex  operations  during 
an  actual  manned  launch. 
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gure  2.  Shuttle  Launch  Equipment  Test  Facility  at  KSC  - Phase 


Figure  3.  Shuttle  Launch  Equipment  Te*t  Facility  at  K.SC  - Phate  II 


Figure  4.  Shuttle  Launch  Equipment  Test  Facility  at  KSC  - Phase  III  (Operational) 


Figure  5.  TSM  and  ET  GH2  Vent  Line  Access  Arm  - 
tals  (TSM  and  RMS)  and  Tower  Simulator 
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A PROCEDURAL  PROPOSAL  FOR  RELATING  TRAINING  DEVICES  TO  JOB  SPECIFICATIONS 


LOWELL  C.  YARUSSO 

Naval  Education  and  Training  Support  Center,  Atlantic 


For  a number  of  years,  the  related  problems 
of  establishing  training  device  requirements 
and  evaluating  the  ability  of  a device 
to  deliver  the  improvements  in  student 
performance  expected  of  it,  have  been  widely 
discussed.  The  Chief  of  Naval  Education 
and  Training  Support  (CNETSUPPORT)  Instruction 
1551.5  identifies  approved  procedures  for 
conducting  a training  situation  analysis 
to  establish  the  training  specifications  and/ 
or  military  characteristics  (MCs)  of  a device. 
Efforts  to  develop  and  systematize  Transfer 
of  Training  Effectiveness  Evaluation  (TTEE) 
procedures  have  already  been  documented* 
and  are  continuing  with  technical  reports 
expected  shortly.  Despite  these  and 
related  efforts,  there  appears  to  be  with- 
in the  device  procurement/validation 
process,  a noticeable  paucity  of  effort 
directed  toward  the  goal  of  assuring  that 
a device  fills  a viable  fleet  need.  And 
yet,  within  the  available  literature,  there 
exists  a common  thread,  specific  behavioral 
objectives  (SBOs),  which,  properly  recognized 
and  seized  upon,  will  yield  a systematic  ap- 
proach and  provide  documentation  of  each 
device's  validity  in  terms  of  the  needs  of 
the  fleet.  Recently,  Naval  acceptance  of 
the  Interservice  Procedures  for  Instruction- 
al Systems  Development  ( IP ISD ) , NAVEDTRA 
106A,  has  added  new  and  consolidated  famil- 
iar procedures  for  systematic  curriculum 
development  and,  hence,  for  coordinated 
device  procurement. 2 

Historically,  training  devices  have  been 
requested,  designed  and  verified  with  scant 
input  from  curriculum  development  activ- 
ities. Military  characteristics  form  the 
base  line  requirements  for  a device.  While 
Naval  Training  Equipment  Center  Instruction 
3910.4,  Military  Characteristics;  instruc- 
tions and  responsibilities  for,  of  1 July 
1969  does  refer  to  training  objectives, 
a review  of  military  characteristics  for 
randomly  selected  devices  indicates  that, 
in  practice,  this  has  not  been  interpreted 
to  refer  to  specific  behavioral  objectives. 
Rather,  objectives  for  the  device  haue 
been  identified  and  documented.  It  would 
appear  that  clarification  and  amplifica- 
tion of  the  need  to  include  behavioral 
objectives  in  the  military  characteristics 
document  is  necessary. 

As  Lawrie  and  Boringer  point  out^  a typical 
sequence  of  events  leading  to  the  imple- 
mentation of  training  often  lacks  validity 
because  failure  to  specify  what  is  needed 
makes  the  attempt  to  determine  what  the 


results  are  a subjective  effort  at  best. 
Usually,  this  is  due  to  the  inability  of  the 
requestor  to  provide  complete  documentation 
and  full  explanation  of  the  training  re- 
quirements, i.e.,  the  training  objectives 
and  the  job  tasks  upon  which  they  are  based, 
which  necessitated  the  initiation  of  a 
request  for  device  procurement.  This  situa- 
tion exists  despite  acceptance  of  the 
philosophy  that 

"The  value  of  any  land,  surface, 
subsurface  or  flight  training 
equipment  . . . must  rest  on 
the  suitability  of  its  design 
for  a particular  training  ap- 
plication and  the  appropriate- 
ness of  the  manner  in  which  it 
is  used. "4 

With  the  adoption  of  the  IPISD  methodology, 
the  potential  exists  to  objectively  "study 
a training  system  as  a whole,  including 
curriculum,  training  personnel,  training 
population  and  training  environment  . . . 

."5  This  study  will  provide  future  con- 
tractors with  specific  information  concern- 
ing both  the  expectations  for  the  device 
and  the  process  which  will  be  used  to  ob- 
jectively evaluate  that  device  in  terms  of 
the  identified  expectations. 

Two  major  alterations  will,  however,  have 
to  be  made  in  the  procurement  process  if 
the  benefits  of  systematic  course  design 
are  to  accrue  to  the  design  of  training 
devices.  The  first  is  the  addition,  to  the 
Program  Manager's  staff,  of  Education 
Specialists  or  other  personnel  who  are  fa- 
miliar with  the  IPISD  process.  The  second 
is  the  inclusion  in  the  contract  of  re- 
quirements that  the  device  under  considera- 
tion reflect  documented  and  acceptable 
expectations  for  student  performance  as 
Identified  by  specific  behavioral  objec- 
tives. In  cases  where  a curriculum  does 
not  exist  at  the  time  of  the  device  request, 
these  requirements  should  be  related  to  the 
job  task  inventory  for  which  students  are 
being  trained.  In  either  case,  evaluation 
would  include  a comparison  of  job  perform- 
ance of  students  who  received  training  on 
the  device  with  those  who  had  not  had  such 
training. 

Taking  the  first  proposed  modification  to 
the  procurement  process,  it  is  apparent 
that,  in  order  to  ensure  full  understanding 
of  the  specific  education  and  training 
requirements  addressed  by  the  device  re- 
quest, input  from  personnel  fully  cognizant 
of  the  IPISD  process  will  be  vital.  While 
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curriculum  development  expertise  has,  on 
occasion,  been  a part  of  past  efforts  to 
develop  devices,  this  input  has  not  been 
routinely  sought  at  the  conceptual  design 
phase  nor  has  it  always  been  sought  at  any 
point  in  the  procurement  process.  Addition- 
ally, when  the  input  of  curriculum  develop- 
ment experts  has  been  sought,  it  has  often 
been  for  purposes  other  than  the  determini- 
nation  of  the  most  effective  means  of 
integrating  a proposed  device  into  the  total 
training  system.  The  results  of  this  over- 
sight have  not  necessarily  been  poor  or 
ineffective  devices;  they  have,  however,  at 
times  been  devices  of  questionable  benefit 
to  the  training  mission  of  the  requesting 
command  and,  therefore,  to  the  needs  of  the 
fleet. 

At  present,  a Program  Team  must  include  an 
Engineering  Member,  an  Integrated  Logistics 
Support  Member  and  a Human  Factors  Member.® 

By  requiring,  in  addition,  that  the  Program 
Team  include  a member  representing  IPISD 
expertise,  the  likelihood  that  the  conceptual 
design  of  the  device  will  be  reflective  of 
identified  training  needs  and  of  the  current 
state-of-the-art  in  educational  technology 
will  be  greatly  increased.  The  IPISD  spe- 
cialist will  be  able  to  bring  to  the  Program 
Team  a depth  of  knowledge  concerning  the 
process  of  curriculum  development  which  is 
not  available  from  other  sources.  This 
knowledge  should  be  an  integral  part  of  any 
effort  to  establish  device  characteristics 
because  during  this  phase  the  relation- 
ship between  the  device  and  the  course  con- 
tent must  be  established.  To  date,  the 
design  process  has  not  always  been  able  to 
document  the  closure  of  this  portion  of  the 
development  loop.  Doing  so,  through  the 
inclusion  of  IPISD  expertise,  will  greatly 
enhance  the  total  device  procurement  process 
and  make  the  end  product  of  that  process  a 
more  complete  and  fully  documented  reflection 
of  the  requestor's  needs. 

The  second  change  suggested  will  be  of  in- 
creasing value  as  the  IPISD  approach  to 
curriculum  development  is  more  fully  imple- 
mented. Once  a curriculum  is  established 
according  to  the  procedures  outlined  in  NAV- 
EDTRA  106A,  it  will  be  of  the  greatest  im- 
portance to  reflect  the  output  of  that  de- 
velopment effort  in  the  design  of  training 
devices.  By  requiring  that  the  contractor 
design  toward  student  attainment  of  specific 
behavioral  objectives  as  reflected  in  the 
Military  Characteristics,  the  value  of  the 
device  will  become  readily  apparent  to  all, 
regardless  of  their  degree  of  familiarity 
with  the  curriculum,  the  device  or  the  job 
to  which  both  are  related.  Additionally, 
the  contractor  will  be  provided  with  more 
specific  goals  which  will  allow  him  to  es- 
timate the  initial  cost  of  the  device  with 


greater  accuracy  and  against  which  his 
performance  can  be  evaluated  with  greater 
objectivity. 

The  IPISD  approach  requires  that  curriculum 
designers  begin  with  an  identification  of 
what  is  and  what  should  be  the  practice  in 
operational  units.  It  further  requires  full 
documentation  of  the  steps  which  lead  from 
the  needs  of  the  job  to  the  content  of  the 
curriculum.  A natural  extension  of  this 
process  is  the  continued  documentation  of 
the  steps  which  lead  from  the  curriculum  to 
a device  request  and,  ultimately,  to  the 
device  itself.  Full  realization  of  the  goals 
of  IPISD  in  the  design  of  curricula,  then, 
will  yield  a pool  of  information  which  will 
permit  the  development  of  devices  directly 
related  to  the  jobs  performed  in  the  fleet. 
That  this  requirement  will  prove  advantag- 
eous is  readily  apparent.  The  difference 
between  the  accepted  practice  and  that  sug- 
gested by  this  paper  is  one  of  specificity. 
The  following  sample  format  should  suffice: 


SBO 

Given  . . . 

The  student  will 
be  able  to  inter- 
pret sonar  pre- 
sentations of 
whales  without 
error. 


MC 

Full  reproduction  of 
sonar  return  from 
whales  with  degrees 
of  background  noise 
ranging  from  1 to 
125%  of  normal . 


As  can  be  seen,  there  exists  in  this  exam- 
ple, a direct  relationship  between  the  mili- 
tary characteristics  and  a specific  behav- 
ioral objective  taken  from  a hypothetical 
curriculum.  In  practice,  a given  military 
characteristic  may  be  derived  from  a number 
of  objectives.  Conversely,  a single  objec- 
tive may  be  part  of  the  justification  for  a 
number  of  military  characteristics.  Never- 
theless, the  end  result  should  be  a con- 
ceptual design  for  the  device  which  makes 
those  interrelationships  obvious.  With  this 
documentation  in  hand,  there  will  exist  a 
straight  line  of  identifiable  needs  from  the 
device  as  it  exists  on-site  to  improve 
training  to  the  fleet  user  whose  personnel 
require  training  to  improve  job  performance. 

While  it  has  long  been  recognized  that  " . . 
. a systematic  approach  to  evaluation  (of 
training)  can  move  in  the  direction  of 
exactness  only  if  it  is  related  to  specific 
objectives.  . little  has  been  done  to 
ensure  the  application  of  this  principle  to 
training  devices.  The  approach  herein  sug- 
gested will  ultimately  permit  the  mili- 
tary training  community  to  take  a more 
active  role  in  the  establishment  of  accept- 
ance test  criteria  through  the  logical  ex- 
tension of  the  required  performance  stand- 
ards (contained  in  Specific  Behavioral  Ob- 
jectives [ SBOs ] ) to  the  analysis  of 
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device-related  student  behaviors.  (It  should 
be  pointed  out  that  this  statement  is  not  in- 
tended to  imply  that  present  considerations 
of  device  reliability,  flexibility  of  design, 
etc.  , should  be  overlooked  or  even  relegated 
to  a position  of  secondary  importance.  It 
is,  however,  intended  to  suggest  that  there 
is  a broad  area  of  concern,  i.e.,  student 
performance,  which  is  not  given  the  status  it 
should  have  in  the  area  of  acceptance  test- 
ing.) The  dual  impact  oi  the  inclusion  of 
curriculum  design  expertise  on  the  concep- 
tual design  team  and  the  marriage  of  military 
characteristics  to  specific  behavioral  ob- 
jectives may  finally  yield  a closed-cycle 
beginning  and  ending  with  the  identifiable 
needs  of  the  fleet. 

There  are,  of  course,  other  benefits  to  be 
derived  from  such  an  approach.  Reliance  on 
fully  documented  specific  behavioral  objec- 
tives in  establishing  device  requirements 
may  well  lead  to  cost  savings  in  several 
broad  areas.  In  the  design  of  a device,  any 
effort  to  limit  complexity  should  certainly 
lead  to  decreased  initial  procurement  cost. 

By  limiting  the  proposed  functions  to  those 
which  are  demonstrably  a part  of  the  curric- 
ulum and  no  others,  the  design  problems  re- 
sulting from  overly  sophisticated  hardware 
and  software  can  be  substantially  reduced. 

It  is  further  apparent  that,  to  a point, 
maintenance  costs  and  downtime  for  the 
device  will  be  reduced  through  function  sim- 
plification. 

Another  likely  advantage  is  that  of  reduced 
personnel  requirements  in  a number  of  catego- 
ries. If  students  obtain  course  objectives 
more  quickly,  the  training  pipeline  is  short- 
ened, allowing  for  greater  student  throughput 
within  existing  facilities  with  consequent 
reductions  in  the  cost  of  training  each 
individual.  If  the  device  aids  in  presenting 
vitally  important  subject  matter  more  effec- 
tively, instructors  are  freed  to  return  to 
operational  billets.  And,  if  the  device  is 
easier  to  maintain,  personnel  are  freed  from 
the  school  house  and  made  available  to 
improve  fleet  readiness. 

The  mechanics  for  implementing  these  changes 
already  exist  and  their  routinization  will 
involve  little,  if  any,  additional  cost  to 
the  present  process  of  device  procurement. 

The  planning  conferences  at  which  IPISD  in- 
puts are  vital  are  presently  held  on  a 
scheduled  basis.  The  training  objectives  to 
which  the  device  should  be  oriented  either 
already  exist  or  will  be  developed  independ- 
ent of  the  device  procurement  process  through 
the  implementation  of  IPISD  methodology.  In- 
formation on  student  performance  is,  or 
should  be,  available  and  will  continue  to  be 
gathered  by  the  requestor  allowing  for 


objective  analysis  of  the  device's  effec- 
tiveness without  increased  workload  or  cost. 
Finally,  implementation  of  the  techniques 
developed  in  the  Interservice  Procedures 
for  Instructional  Systems  Development  will 
provide  a continually  expanding  data  bank 
which  will  facilitate  verification  of  the 
applicability  of  training  devices  to  job 
performance. 

Whether  or  not  this  proposal  will  yield  the 
predicted  benefits  can  only  be  determined 
through  a practical  application  of  the  ap- 
proach presented.  The  most  logical  first 
step  would  be  to  choose  a minor  device 
request  requiring  development  of  a new 
training  device  which,  while  unique,  is 
similar  in  many  of  its  major  parameters  to 
an  already  existing  device.  By  comparing 
the  end  result  of  a procurement  process 
based  on  specific  behavioral  objectives 
with  one  not  so  based,  determination  of  the 
effectiveness  of  this  proposal  can  be  made. 
To  ensure  that  such  determination  is  effec- 
tive and  objective,  the  use  of  the  expertise 
of  such  an  organization  as  the  Training  An- 
alysis and  Evaluation  Group  (TAEG)  in  con- 
ducting comparative  studies  of  the  train- 
ing effectiveness  of  the  two  devices  should 
be  considered.  If,  in  this  case,  the  ap- 
proach suggested  by  this  paper  proves  of 
substantial  benefit,  another  trial  on  a 
more  sophisticated  device,  such  as  one  of 
the  14A2  series  or  a flight  simulator,  could 
then  be  undertaken.  Eventually,  the  ap- 
proach could,  if  successful,  be  applied  to 
all  device  procurement  efforts. 

In  summation,  present  efforts  to  identify 
the  validity  of  device  requests  and  to 
evaluate  the  effectiveness  of  the  device 
within  the  total  training  setting  recognize 
the  necessity  to  relate  procurement  and 
evaluation  to  specific  behavioral  objec- 
tives. A systematic  approach  which  will 
allow  curriculum  developers  to  enter  the 
procurement  cycle  at  the  conceptual  design 
phase  is  needed.  Their  input  will  help 
ensure  the  training  relevance  of  the  con- 
tract specifications.  Further,  the  contract 
itself  must  reflect  the  recognized  place 
of  specific  behavioral  objectives  in  deter- 
mining training  requirements  and  in  estab- 
lishing evaluative  criteria.  Among  the 
potential  benefits  of  such  a system  are: 

a.  Improved  contractor  performance 
through  increased  understanding  of  the  goals 
envisioned  for  the  device. 

b.  Potential  savings  across  a broad 
spectrum  of  cost  areas. 

c.  Increased  fleet  readiness  through 
more  job-specific  device  requirements. 
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By  beginning  now  to  base  development  of  devices 
directly  on  specific  behavioral  objectives, 
the  Navy  can  ensure  that  its  training  devices 
will  relate,  through  the  curriculum  they  are 
a part  of,  to  student  performance  both  in  the 
school  house  and  in  the  fleet.  This  rela- 
tionship will  ensure  that  training  devices 
support  the  needs  of  the  fleets  and  the 
overall  mission  of  the  Navy,  a necessary 
step  toward  total  readiness  afloat. 
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